querysub 0.433.0 → 0.436.0

package/spec.txt

@@ -1,1193 +1,1193 @@
-
-
-Trigger values
-    1) validStateComputer.ingestValuesAndValidStates
-    1.5) authorityStorage.ingestValues
-    2) authorityStorage.ingestValues
-    3) lockWatcher2.watchValueLocks
-    4) validStateComputer.computeValidStates
-    5) lockWatcher2.getValuePathWatchers
-    6) pathWatcher.triggerValuesChanged
-    7) pathWatcher.triggerLatestWatcher
-    8) PathValueControllerBase/clientWatcher.localOnValueCallback
-
-Source of values
-    1) clientWatcher.setValues
-    2) PathValueCommitter.commitValues
-    3) (predictions) validStateComputer.ingestValuesAndValidStates
-    4) (non-predictions) PathValueControllerBase.createValues
-    5) validStateComputer.ingestValuesAndValidStates
-
-Archiving values
-    1) authorityStorage.ingestValues
-    2) pathValueArchives.archiveValues
-
-TODO: pathValueAuthority detection
-
-
-Core parts
-    - Valid evaluation
-        - Needs external value syncing, which can give it all past values, and all future values for a path
-    - Value syncer
-        - Needs someone to give it authorities, and tell it when an authority goes down (at which point it finds a new authority and syncs the new values)
-    - Authority tracker (supports registering, checking, notifying, and generally synchronizing all authority information between all authorities)
-        - No requirements
-    - Authority setuper
-        - Needs authority tracker
-        - Needs value syncer
-    - Authority (stitches together functionality)
-        - Needs authority tracker
-        - Needs authority setuper
-        - Needs value syncer
-        - Needs valid evaluation
-    - Disk value saving/retrieval
-
-FunctionRunner is funny. It wants a full history, for reading, but does not calculate the valid states of PathValues.
-
-
-Optimization / optional features list
-    Valid Evaluation
-        - Only need to sync some data, not all data. 
-        - event: sometimes we discard data
-        - isTransparent: Sometimes, even though two values are different, they can be considered equivalent. 
-        - gc: Older values won't change their valid state, so we can discard their locks.
-    Value Syncer
-        - Path watches
-        - Parent path watches
-        - Authority path (tree) watches
-        - Valid watching (authorities know this, so it's free to also sync this)
-        - Only latest/valid state watches (no history, just latest, more efficient)
-        - Special connection clobber logic (you can't store the history after you disconnect if you are not the authority)
-        - Special logic
-        - Reused for internal watches (rendering, etc)
-    Authority tracker
-        - Immutable authorities, to simplify usages
-        - Path requests => authorities might take slices from multiple authorities
-    Authority setuper
-        - Once on startup, so we don't change authorities, to simplify implementation
-        - Leverage disk snapshots to make loading faster and lighter for existing nodes
-    Authority
-        - The stitching together of all the values is reused by the function runner in order to rerun functions when they are invalidated
-    Disk value saving/retrieval
-        - Only old values with discarded locks are written, and the history is compressed occasionally 
-
-
-
-
-
-*** Goal is to get to === Full public website stack milestone === ***
-=== 3 tasks to fully fledged syncing KVP database, 2023 / 25 / 2 ===
-=== 1 tasks to fully fledged syncing KVP database, 2023 / 26 / 2 ===
-=== FINISHED fully fledged syncing KVP database, 2023 / 28 / 1 ===
-=== 6 tasks to "Full public website stack", 2023 / 31 / 1 ===
-=== 5 tasks to "Full public website stack", 2023 / 31 / 1 ===
-=== FunctionRunner bones done!, 2023 / 31 / 2 ===
-=== Disk garbage collection bones done, 2023 / 11 / 3 ===
-=== Access for non-network nodes (via permissions) done, 2023 / 19 / 3 ===
-=== Clientside + clientside function prediction done! 2023 / 23 / 3 ===
-=== Prototype SD app with 2 creation modes, arbitrary model support, paint, diffuse, openpose, onnx model generation, etc. 2023 / 10 / 1 ===
-=== Lots of failed embedding training research. 2023 / 12 / 18 ===
-=== Better single source of truth. 2023 / 12 / 31 ===
-=== Better serialization format. 2023 / 12 / 31 ===
-=== Fixed storage / multi process storage. 2023 / 12 / 31 ===
-
-TIMING: Transactions, currently about ~1.5ms per transaction + 10us per write and read path in the transaction
-    - About 5ms latency, the fast times are only if you don't wait for transactions to finish (because why would you wait when the whole system is based on predicting writes?)
-    - With 1 watcher (with 0 watchers transactions are probably about 0.5ms per)
-    - Much of the transaction over is serialization time, mostly in JSON.stringify (to send the value to the watcher)
-    - MUST FASTER NOW, if they are batched
-
-TIMING: Function calls, about ~1ms-3ms for a trivial function, if it can be batched with similar functions
-    - Slower if we need to sync new paths
-
-
-Fixed function props / function closure parsing
-    - Presently, if a parent re-renders, and it passed a lambda to a component, that component will always have changed props. HOWEVER, this is very inefficient!
-    1) Process the file to convert lambdas to provide debug information which includes all of the values they close upon (as properties set on the lambda itself). Also an id, unique to that specific piece of code.
-    2) When evaluating if props changed, if the lambda id is the same, and the closed upone variables are ===, then... it's the same thing, so consider the prop unchanged
-    3) Test if all of our "updateOperation" lambdas in SideOpConfig.tsx work with this. I THINK updateOperation is the same, but if it isn't... we can always recognize nested lambdas.
-
-Option to not use permissions checks locks
-    1) Create a way to have any code run without locks (we might already have this?)
-    2) Add a flag in schemas to either specify all functions, or certain functions should run permissions without locks.
-        - Also something to specify a function SHOULD run permissions with locks
-    - Using the ArchiveViewer verify "heartbeat" goes from 25 to 0 locks
-    - If you step into pathValueClientWatcher.ts:setValues in FunctionRunner, you can see the actual paths that are used.
-    - Benchmark a simple function to see how much faster this is, and how much less memory we use
-        - If it doesn't matter... maybe remove the feature?
-
-More corruption resistance files
-    - Add a new serialization format, via versioning in the settings
-    - Each datum will be length prefixed WITH a special sentinel AND with a checksum for the datum data
-        - The sentinel can depend on the length (but otherwise be constant)
-    - We can have a recovery mode (automatically attempted if the checksums are wrong?) where we scan the file for length+sentinel pairs (where the sentinel matches the checksum), see which ones have a valid checksum, and then use an algorithm to decide which ones to use
-        - Maybe we sort by success rate, which is [-numberOfValidChecksumsOverlapped, length], which is very likely to work EVEN if there are values which have the right checksum, but are actuall corrupted.
-            - As long as the checksums are random enough, so... we should never use any small values for checksums, etc.
-    - Record the value layout in the settings, so we can generate arbitrary values, even if we've never seen that layout before.
-        - A series of parallel object arrays, with each object having flags (which indicate which values exist), values which always exist, etc
-            - Value types will be string, float64, byte, Buffer[]
-                - and... we might as well add support for short, int, float, and uint (uint is a good way to store a guid, via storing 8 uint variables).
-
-Schema/binary PathValues accesses
-    PER Path, not per value. Values will likely be lazily deserialized anyways, making them either pointers, or just numbers, stored in unchanging objects, which... should be quite fast.
-    0) First our stream/schema wrappers. Which at first just convert to raw path reads/writes, but once we add schemas they can efficiently create/use those
-    1) PathValue.path (string[]), need to be converted to an opaque type `Path`
-        - Helpers to manipulate it, and go back to string[], string, etc.
-    2) Have Path be able to store itself in schema + variables mode
-        - Where schema is shared across all Path instances
-        { schema: { path: (string|Variable)[] }; variables: (string | number)[]; }
-        - Add functions to access the schema and variable values
-    3) Create an efficient `Map<Path, T>`, via using schemas from the paths
-        - By requiring Path as the input we can directly use the schema to narrow down the cases, and then within that we just have to lookup values by variables
-    4) Use Path everywhere, replacing PathValue.path, using our efficient lookup to manage it.
-        - I think a lookup should handle all the cases. We should be able to nest them as well?
-    5) This SHOULD let us entirely get rid of path joining, which should be SO much faster.
-    6) Update serialization to keep schemas, instead of converting them back to paths. This should be more efficient, and a lot smaller.
-        - Values serialization won't change, and we still need to encode the variables, but... it should still be a lot faster.
-    7) Try to remove as many Path conversions to string (and also to a lesser degree string[]), as possible, by making them schema aware.
-    8) Investigate further optimizations
-        - Replacing variables with numbers, so we our internals Maps can be replaced with arrays / typed arrays
-
-    IMPORTANT! Actually... a lot of the following is wrong. We should have this PER atomic value, and perhaps ONLY for paths!
-
-    Base code
-        reader
-            let viewTime = 0;
-            for (let user of global().users) {
-                viewTime = user.viewTime;
-            }
-            return viewTime;
-        writer
-            for (let [video, viewTime] of viewedVideos) {
-                global().users[userId].viewTime += viewTime;
-            }
-
-    Schema optimization
-        writer
-            let changeStream = new SchemaPath(() => global().users[wildcard].viewTime);
-            write(() => {
-                for (let [video, viewTime] of viewedVideos) {
-                    changeStream.write(userId, viewTime);
-                }
-            })
-        reader
-            let schemaWatcher = new SchemaPath(() => global().users[wildcard].viewTime);
-            watch(() => {
-                let viewTime = 0;
-                for (let [value, userId] of schemaWatcher.getValues()) {
-                    viewTime += value;
-                }
-                return viewTime;
-            });
-        reader specific path
-            // I guess we have to support gets as well. It should still be faster, as the schema is pre-defined
-            let viewTimeWatcher = new SchemaPath(() => global().users[wildcard].viewTime);
-            let factorWatcher = new SchemaPath(() => global().users[wildcard].factor);
-            watch(() => {
-                let viewTime = 0;
-                for (let [value, userId] of schemaWatcher.getValues()) {
-                    let factor = factorWatcher.get(userId);
-                    viewTime += value * factor;
-                }
-                return viewTime;
-            });
-        delta reader (eventually)
-            // This can easily be extrapolated to just getting deltas
-            let deltaWatcher = new DeltaWatcher(() => global().users[wildcard].viewTime);
-            let viewTime = 0;
-            onDelta(() => {
-                for (let [newValue, prevValue, userId] of deltaWatcher.getChanges()) {
-                    viewTime += newValue - prevValue;
-                }
-                return viewTime;
-            });
-
-    - Can be binary, or not.
-    - Streams can work with non-streams, and vice versa.
-        - The streams need to be setup in a tree, so we can efficiently check for watchers of them
-    - We also need to support partial key watching. Often we will watch a few keys (and then within them, maybe all keys at another level).
-
-    1) START by supporting write streams (but NOT read schemas), as this allows us to define our schemas.
-        IMPORTANT! Actually... a lot of the following is wrong. We should have this PER atomic value, and perhaps ONLY for paths!
-        - This will give us a big chunk, which we will pass around (even passing around arrays of chunks).
-            - The core will break this apart somewhat, with an object per schema, and then a tree of maps for the dynamicValues inside of it (and global lookups for the locks and values)
-        - We will never create this from PathValues, instead, we will append values to specific schemas as we build it
-            - And... locks need to be kept track of as well
-        - So we need a global "captureWrites", to set the state
-            - It will return a function, which will take parameter to finish the writes?
-                - Or... something. We need to look at proxyWatcher and see what the best way to do this is. I think it might check the values before finishing them?
-        - The schema builder functions (on the object returned by defining the schema), will then internally add to a lookup in the globally capturing state (keys by the schema seqNum)
-            - Directly adding to the PathValue for that schema
-                - A lot of fields will only be set on finish
-        - I think we still need to support the helper flags (.valid). Which is actually fine, even if it's binary. It's fairly easy to flip flags in binary data...
-        - When we need to reason about PathValues independently we can provide some kind of pointer than can be used in conjuction with the chunk
-            - Although most of the time we will just provide iterators to iterate over all the PathValues?
-            - I guess eventually the pointer could be serializable too, so that we could select PathValues with a Chunk + Buffer containing pointers, so we entirely manage our own memory. But... probably not for a while, as { schemaId: number; index: number }[] should be VERY efficient to allocate and store, especially if it isn't persisted.
-        // PathValue
-        {
-            schema: ({
-                type: "constant";
-                key: string | number;
-            } | {
-                type: "dynamic";
-            })[];
-            values: {
-                dynamicValues: (string | number)[];
-                // Pointer to position in value Buffer
-                setValue: Pointer;
-                // SeqNum to ReadLockGroup (as often many values will have the same set of ReadLocks)
-                readLocks: Pointer;
-                ...more fields for various PathValue fields
-                // NOTE: ReadLocks need to reference another binary structure
-                //  - Probably via a locally unique seqNum, which is remapped upon receiving data over the network
-            }[];
-        }[]
-        // ReadLockGroups
-        {
-            seqNum: number;
-            lockSeqNums: {
-                schemaSeqNum: number;
-                seqNum: number;
-            }[];
-        }[]
-        // ReadLocks
-        {
-            seqNum: number;
-            schema: ({
-                type: "constant";
-                key: string | number;
-            } | {
-                type: "dynamic";
-            })[];
-            locks: {
-                seqNum: number;
-                dynamicValues: (string | number)[];
-                startTime, endTime, flags, ...;
-            }[];
-        }[]
-        // Values
-        {
-            seqNum: number;
-            // arbitrary binary data
-        }[]
-    2) Isolate this object structure, so it is only manipulated or read via helpers, so we can change it to use a binary format later on
-        - This means adding functions to go to/from binary, even at the start, and just having it go to PathValue and use PathValueSerializer for now.
-        - We will have unique id (a number, locally unique) per schema, so we can very quickly tell if anything might match
-        - OH! use global objects for schema defs, so equivalent schema defs share an object. This lets us store flags for them!
-            - Not the whole schema, just the def part
-            - Just a pointer as well
-            - If we have conflicts with other schemas (now or in the future), this needs to point to the shared data for the combined schema.
-        - Expose the shared objects as createdSharedAccessor<Object>(key) => { get(schema); set(schema, value) }
-    3) THEN, use this structure throughout, replacing PathValues in the entire application with lists of these chunks
-        - We'll probably store in the core in an entirely splayed manner. Maybe the schema, then a tree of dynamic values with the leaves being the set value?
-            - The set value will contain much of the PathValue, excluding the path
-        - Update the core code to store data in schemas
-            - If schemas conflict, we take the more general union of them. Annoying, but this should be rare, as schemas aren't recursive.
-                - Ex, "x.*.z" and "x.y.*" NEED to be combined to "x.*.*", at least for storage.
-                - AND, we need to store the original schemas, and have them MAP to the combined schema!
-            - So we end up with a non-schema data storage, and a bunch of schemas
-            - TWO ways to access data in schemas
-                - If you have a schema, you just directly lookup your data in the shared schema data, which is owned by the core, but every schema points to it
-                - If you don't have a schema...
-                    - We need a lookup of schemas as values with wildcard (I guess empty keys equals a wildcard? Hmm...)
-                        - I think we disallow reading empty keys? If not... maybe we should? They are already used a wildcard watches anyways...
-                    - Check each part of the path, with wildcard filling the rest "x.y.z" => ["*.*.*", "x.*.*", "x.y.*", "x.y.z"]
-                        - OH! That's not enough. Because... "x.*.z" is a valid path, which won't match any of them.
-                        - So... maybe a sorted list?
-                    - Should be reasonably fast...|
-    4) Then update it to be binary
-        - On write
-        - And on network traversal
-        - I think PathValueSerializer needs to be rewritten. Basically, it will be given schemas ahead of time, sometimes (or maybe always, as we might as well always encode in our base schema), and then encode the paths more efficiently.
-        - OH! ReadLocks need some kind of schema as well. Hmm... I guess, they can use the schema of their readers?
-    5) ALSO, create some kind of global string lookup? A lot of string keys will be the same, and it's easier to manipulate/store numbers
-        - We will need to provide definition when we send them over the wire though
-        - THIS is IMPORTANT! Without this our strings become inefficient, and the fastest code would involve mapping all strings (such as userIds), to numbers at an application level, which is a lot of work that the framework should really handle...
-        - Maybe not for ALL string keys? Hmm...
-        - OH! And... they only need to be unique per schema? Hmm...
-        
-        - Test with structures like "components.*.x" (but lots of different x values)
-            - Hmm... the big speed issue is that the component ids don't map to an array. Although, in theory, we could do that, remapping ids to indexes. And then... reuse indexes on gc, so they become direct offsets into binary memory.
-                - Then... I guess we can remember the last mapping, and so quickly go from string to index, with just a === on the strings (which should just be a pointer comparison), which... and even though the index is dynamic, that's still basically just a pointer. So... we could recover a lot of the speed
-            - Maybe we should try this out in a mini-benchmark, with a greatly simplified structure (values constant size, no extra fields, all schemas are just 3 long and have similar structure)?
-        - AH! Have the schema context wrapper (so schema reads/write get registered), give us a context object, which it gives back for the same type of access.
-            - THEN, we can have state per watcher, and... maybe per watcher+schema. Then we can use this to cache the last keys at indexes per schema, per watcher... and for component accesses... this will always be the same, making the mapping instant every time
-    6) Support reading from schemas as well
-        - If the data is already stored in the schema format, querying in this format should be very fast
-        - We will need a shorter read binary format, which has a schema and dynamic values
-    7) AUTOMATIC schema definition generation from createSchema (replacing the code), and replacing (where possible) the uses of the schema to use the schema definition
-        - Probably the hardest part, but... the most important. Without this our application code becomes too difficult to use
-            - Nested accesses become difficult, but... not impossible.
-            - Object assignment needs to become global
-                Local assignment
-                    let list = data().list;
-                    for (let datum of newData) {
-                        list[datum.key] = datum.value;
-                    }
-                Global assigment
-                    let list = data().list;
-                    for (let datum of newData) {
-                        data().list[datum.key] = datum.value;
-                    }
-                Schema based
-                    let list = data().list;
-                    for (let datum of newData) {
-                        listSchema.write(datum.key, datum.value);
-                    }
-
-                Reads are more likely to use highly nested assignment... even cross function
-                    - Cross function stuff is hard.
-                    - We might need to specialize functions?
-                        - As in, per type of possible schema input object, make a new function, and still leave a non-schema function.
-                        - Actually... this should be fairly easy. We aren't going to use === on the functions... ever? And if we do... then I guess it'll fail (or they can mark them as non-specializable, or... we can detect if === is ever used on the function).
-
-    8) Delta watchers
-        - If we store change reasons in the schema format... then we can query deltas for free
-            - Stored in the packed format.
-
-Global nested object state for latest values
-    - Can have multiple global objects, maybe one per collection / schema?
-    - We still need to register reads
-        - If we can get code to not write duplicate reads (which a lot of code can be verified to not do anyways), then tracking the reads becomes very simple with our binary schema data, basically just involving writing numbers (or even just the start/end of ranges) to an output Buffer.
-
-Code transformation for accesses
-    - Add code which can parse a function, find any schema accesses (this requires walking the type tree, and maybe checking other files), and transform the code to call data.q() functions instead (if the schema is a schema2 schema)
-        - Add support for __callerFileName__, etc so this function can determine the caller file, so it can know how to parse it.
-        - At first do this via a stand-alone function call?
-            - And then make it a parameter on the schema?
-        - OR, should we transform the code directly?
-        - If we do it at runtime, we have to figure out capturing scope again. This is hard, and breaks static variables, but... maybe that's okay?
-            - We could always warn about static variables
-            - For any non-static variables (constants, functions, etc), we will need to duplicate them, and anything they access, etc).
-                - THIS is actually REALLY nice. We can add a file name for this so it can be discoverable in devtools, or, the user can set `debugger;`, at which point, they will see a file which just has the function and anything it depends on. This makes debugging easier, as it removes everything not involved in that function (and more importantly, shows everything that IS involved, which could be surprising, as you might miss the fact that certain functions are being called, but if their definition is there you are far less likely to miss it).
-
-Code transformation to turn types into schemas?
-    - If we could do this... we could get fast behavior from normal typescript code
-    - We would need an out in case the code accesses the schema in a non-schema type way (just for accesses, the backend should already be able to handle this).
-
-Binary ValuePaths, which are never decoded
-    1) Allowing accessing a value in a way which returns the intermediate deserialized object (the Buffer[], strings lookup, etc) PLUS the id (or an object?).
-        - Call it, ValuePathBinary
-    2) Allow using ValuePathBinary as an input to SchemaPath parameters, updating the SchemaPath code to directly copy the binary data
-    3) Create a function which exclusively accesses ValuePathBinary, using it for nested sets (getting a value, using that to index another value, etc)
-        - Verify we are never actually decoding the values
-    4) Hack together something where we can NOT decode strings in the ValuePath serializer, sometimes, as our special function will never use strings directly.
-    5) Benchmark to see the benefit (probably via memory pressure), or not decoding the strings?
-
-Automatic commit wrapping of local synced writes
-    - If we write or read from local synced state... and are not in a transaction, we should start a transaction, which exists behind the end of the call, and is only stopped:
-        a) After Promise().finally()
-        b) When any non-local synced state is accessed (which will cause us to throw as well)
-        c) When any transactions are explicitly started
-    - This resolves the issue of having to constantly wrap state writes, which is just annoying
-    - We SHOULD wrap this into a singleton local watcher which we re-use for local events ("once" watchers). This will make events faster, and should make it easier to implement.
-ALSO
-    - Allow accessing local state values without being in a synced state?
-
-Server crash log protection
-    - Up to 10 times per hour, synchronously write unclassified logs to special single log blocks
-    - We will delete these special blocks when we finally write the logs
-    - This way if we get a fatal error, the console.error before we crash (which HOPEFULLY) we get, will let us know what went wrong.
-    - OR... maybe not... maybe the server manager should just watch for crashes and log the most recent stdout/stderr logs?
-
-PathValueController direct remote database support
-    --ext-remote "D:/code/other/loader.ts"
-        {
-            /** undefined means it doesn't need to be loaded remotely */
-            loadRemoteValue(path): Promise<unknown | undefined> | undefined;
-        }
-        We'll probably have a timeout, just providing epochValues if loadRemoteValue takes more than a few seconds
-    We COULD allow for core to be resolved to the same thing independent of version (by making the core register itself globally, and all cores to use the same thing).
-        - AND, we could do the same thing for a few other files, such as clientWatcher, etc
-        - This would allow clientWatcher.setValues to be called explicitly
-        - This MIGHT be useful, but... then again... it adds a lot of complexity
-
-Hot render throttle
-    - We want all inputs to be hot, BUT, to not render on each keydown. So... we need to delay rendering triggering?
-        - BUT, we run into issues if a button then runs, which expects to have the most recent data closed upon? Ugh...
-
-Auto commit mode
-    - If anything is accessed outside of synced state, enter a commit, and end that commit when:
-        a) Promise.resolve().finally
-        b) We try to end any other commit
-    - Should be a Querysub config value which is on by default
-    - VERY useful clientside
-    - We need to support "lazyClose" watcher in proxyWatcher, that DOESN'T reset the runningWatcher UNTIL it is explicitly reset, OR, until another watcher tries to run
-    - We need to detect synchronized accesses in proxyWatcher and start this special type of commit
-    - We need to know if we are in this state, and THROW if we access any non-synced paths
-        - This is mostly for non-local paths
-        - Throwing WHEN they paths are accesses is required, otherwise the error location is too annoying
-
-Watcher diff mode
-    - Running stress tests (satTest.ts) with a single watcher shows that the slowest part is our function watcher, specifically all of the parts that don't operate in a delta mode (ex, setWatches).
-    - If we made a watcher fully support a delta mode (which is fairly easy, as it converts all the changes to deltas anyways), the watcher would likely work MUCH faster.
-    - ClientWatcher needs to expose a delta interface
-    - proxyWatcher also needs a delta mode
-    - Rerun the benchmark after using the for FunctionRunner, and our call/s should go up significantly
-
-Shard display / manual sharding / automatic sharding
-    - Sharding will be fixed on process start
-    - We can augment our database path size distribution display to also support explicit sharding control
-        - This means we have all machines register, and then tell them what pathAuthorities to run (generally just running a single process, but always a new process, with us never changing pathAuthorities for a process/node)
-    - THEN, we can write code that automatically sets the sharding configuration, not changing it too often, but keeping everything fairly well distributed
-        - Our merging should automatically handle cleaning up dead nodes, so we really just need to create and kill processes to control sharding.
-
-Function implementation benchmarking / profiling / diagnostics
-    - Have a debug page which shows all functions that have been run, with diagnostics
-        - Time taken (total, average, etc)
-        - Lock count
-        - Rerun count
-            - Really important, as it is easy to right functions with a high rerun count. Ideally every function should run at most 2 times, once to know which path it requires, and again once those are all synced.
-        - Reject %
-
-Fix wildcard permissions
-    - Wildcard values should do more than just check "", they should check all possible direct permission keys
-        - This fixes the `{ admin: { PERMISSIONS(){ return users()[config.callerId].isAdmin; } } }` type of permission check
-            - Also `{ serviceSecrets: { PERMISSIONS(){ return false; } } }`
-        - It isn't SO bad, as Object.values() only provides shallow values, but... we should still fix it.
-
-Browser local PathValue caching
-    - https://rxdb.info/rx-storage-opfs.html
-    - Do it at a core level
-    - Start with being able to enable it for specific paths, triggered by a flag to createLocalSchema which causes that entire schema to be synced.
-        - Or... createLocalSyncedSchema?
-    - Eventually add support for remote paths, but in a way to not erronously trigger the sync flag
-    - We need to batch the PathValues that we store, and load them in efficiently, etc, etc
-
-Combined console/event display
-    - Everything grouped by event, with minor sorting based on the latest events, but it is mostly just an aggregator
-    - Show the most recent events
-    - Allow drilling into a specific event, searching it, filtering it, etc
-    - Allow setting up debug workflows
-        - Take a specific event / search, run some code on it to filter and create another search, which gets values, which we then can run another set of code on, etc, cascading
-    - Tree summarization / navigation
-        - For tree display, take any node that has > N children (maybe 100?), and wildcard it, collapsing all children into a single node (and merging all child tree, etc, basically rewriting it so instead of many keys, it has exactly 1 key). This should give us a small number of finite paths. Then we can drill down far enough to split each node into large enough chunks (so we aren’t only looking at roots, but also not only looking at children), with the split factor being configurable (so we CAN just look at root, or just look really high level).
-
-Heap analyzer
-    - Start by showing shallow size with a depth of 1, and then allow clicking to add more depth
-        - Maybe allow viewing the options at each level (instead of just taking the first reference)?
-
-Better serverside logging
-    - Batch it or something, so 20 requests per second (which isn't even that much), doesn't cause the console to be unusable.
-    - Maybe... group by type, and show all the categories on the screen at once, allowing the user to type into the console to expand a group, pin a group, filter, etc
-        - The most recent groups can be shown first, WITH, extra prioritization for severity
-        - Expanding a group will show the specific entries (otherwise we just show the a count, and MAYBE the latest value?)
-        - Filtering would probably be useful too
-    - OR... we could just show a really simple log, and link to a web UI?
-        - In the web UI we could show logs for multiple processes too!
-            - We could tag groups with processes that contribute to them
-            - And also show active processes, and recently terminated processes
-
-Cloudflare bootstrap caching
-    - Add round robin DNS entries (or just verify we are already using it), for our root domain, and turn on some level of caching
-        - I think we return the correct e-tag, so we should be able to cache fairly heavily
-        - If we turn the file download to be 2 stage we can cache the second stage forever (including the hashes we obtained from the first stage), which... SHOULD be a lot faster (the first call will be slower, and two calls are required, but the second call should be always cached in cloudflare, and then in the user's browser).
-    - Fix clientside node routing
-        - Update our certificate generation to have one level of wildcard.
-            - I believe (we should verify this), you can't have multiple levels of wildcards, so... this certificate WON'T be able to impersonate our full node domains (which are 3 parts), and even if it can... we do extra verification of the public key, so... it would actually be fine.
-        - Publish A records for our machines
-        - Convert the node ids to just use the machine id clientside, allowing the client to pick any Querysub node
-    - Have the node list accessed via HTTP, and cached for a few minutes, as it will change constantly, but... clients don't need the latest version
-        - I GUESS after the first read, if we can't find a Querysub node, we can add some flag to disable caching for it, which would eliminate the lag between fixing all the servers and the site being usable again
-
-Querysub node sharding
-    - Presently we allow any client to use any Querysub node. We might want to shard them too? Or at least make it allowed?
-
-Archives disk caching
-    - Cache backblaze files on disk.
-    - Only files that are at the file size limit and old enough (coordinate with the thresholds the merger uses).
-        - At a certain point the merger will stop touching files, or touch them less often, and so we will be able to cache them on our disk for a while.
-    - Limit our size to a fixed disk % (probably 5%?), with a configurable list of disks to use
-
-Clientside sourcemaps in error callstacks (maybe just have Querysub install the error stack library by default, in the browser?)
-
-typenode improvements
-    - Log typenode timings after setImmediate, top 10 + remaining timing, in a table
-        - Also, maybe entries with time > 10ms (including remaining), and not logging the table if this would make it empty.
-
-Tool to suggest balanced path ranges (based on data size, not access patterns).
-    - We can even just have it be a function, which we call on startup (telling it the count of nodes, and which index we are), and which caches the shards. It isn't live automatic sharding, but it is technically offline automatic sharding, which isn't too bad.
-
-FunctionRunner automatic balancing
-    - Exposed functions will also define define their shard distribution.
-        - Ex, setShardDistribution(addToUserBalance, (userId, balance) => userId)
-        - It is somewhat important for the shard distribution to be consistent across various functions.
-        - If this is just set on our most commonly called function, sharding should work quite well
-    - IF a shard becomes too far behind, other nodes will pick up the slack, with the closest shards picking it up first, then those farther away, etc, etc.
-        - Hopefully we can use the shard values to prevent too much duplicate work, even without any direct communication between the nodes
-
-Turn on and test backblaze storage
-    - Create a utility to go from disk => backblaze (and might as well go from backblaze => disk)
-    - Test starting a server on another machine
-    - After setting up the .json keys files, it... should just work?
-
-Anti-rejection code (isn't REQUIRED to make the database useful, so we should wait. Would be pretty slick though... Ideally this can all be extensions that have no or only modular impact on the core functions, or even no or modular impact on the proxy? Although that might not be possible...)
-    Summary
-        Excess work (N^2) due to rejections
-            - Past time reading (removes causality guarantee, almost completely preventing rejections)
-            - Unsafe reads (to remove locks, and then selectively add them back)
-        Jitter (due to rejections)
-            - Pase time reading (even further in the past than required, to allow a buffer of values)
-        Jitter (due to varying client lag)
-            - Past time reading
-        Lag makes player near impossible to kill
-            - Adjust writeTime to stay > min value, depending on the values changed (some values have to be < 100ms, etc)
-        Laggy player does things only they can see
-            - Adjust age of past dependencies to stay > min value, depending on the values changed (some values have to be < 100ms, etc)
-        Clientside operation lag resulting in inability to perform delicate operations (ex, shots miss)
-            - Past time reading
-        Clientside operations applied out of order
-            - More consistently adjust writeTimes (instead of one some functions but not others)
-        Operation cause-and-effect no ordered (phasing through objects)
-            - Stop using past time reading
-        Clientside hacking script to automatically react to values in the past
-            - Serverside changing of time, rejecting clientside predictions, AND, something to tween to the correct state, to prevent snapping
-
-    todonext
-        brainstorm: `So what is the FAIREST way to prevent lag from HELPING.`
-
-    == Test app, with top down "shooter"? (with just circles and lines?)
-        - Movement will be "key" based, not frame based. This is harder to implement, but takes a lot of pressure off of the synchronization code.
-            - And... the look position will be... just blurred, using maybe 3 ticks per second? Ugh... that will work, for now. Eventually the synchronization code SHOULD be fast enough to handle 30 ticks per second, but presently... our overhead is just too high, and so operations that SHOULD take ~10ns take ~10us, and there's not much we can do about it until we use deltas everywhere, and replace our usage of proxies.
-            - This makes the physics system harder to calculate. Although, not that much harder, as we can still use the "intersection" approach, if we want to, by locally emitting positions per frame.
-                - I'm not sure the best way to do it... intersections (which really decomposes into line segments) is nice, as it allows infinite precision.
-                - Of course, if we have any concept of "gravity", then it is harder. BUT, even then, we can still extrapolate, and even better, we can extrapolate with non-linear segments, which is really the best way to do those kind of simulations anyways.
-                    - We WOULD require a way to make the non-linear segments update consistently, but... as we have a universal time, that wouldn't be so hard...
-            - WELL, no matter what we need some kind of snapshots, to prevent having to re-simulate TOO much state.
-                - So... we can have "keys" only last for a certain period of time, and after that, movement stops. Of course, if we had gravity this would make it possible to freeze in mid-air, but, eh... it is what it is.
-                    - AND, if we had any NPCs or AI, those would require an app server, or some kind of general purpose interval function calling server anyways...
-    == Add AI to automatically move around nodes, and automatically shoot
-    == Add multiple nodes, with various latencies, which flucuate with different magititudes
-        - We should see nodes jumping around
-        - It will be hard to hit laggy nodes, as you don't know where they are!
-
-    Delayed prediction rejection
-        - Not sure if this is needed, but in theory if there is contention a function might always require multiple runs to be correct. This means our prediction will always be rejected. BUT, it is probably pretty close, so... we should just keep it around for a bit, until FunctionRunner finishes up the function, giving us the most up to date state.
-
-    Past time client reading
-        - Intentionally read client positions older that the latest, in order to cause a smoother state.
-        - Will read a combination of a fraction + value in the past (so clients that are 1000ms +/- 500ms
-            still look like they are smooth)
-        - The readLocks will have to have an empty time range (as we KNOW their range will have invalid values)
-            - At least for write values. For the UI, this doesn't matter, the UI is readonly
-        - Bakes in these values to the function call when asking the server to run the function
-            - When we evaluate the function call, if those paths are read, we have to adjust the read times
-            - OF COURSE, only if those paths are read with the request for "past time client reading"
-        - Allow CUSTOM limiting (we always limit to a few minutes), of time in the past, adjusting the read to be more recent if it is too old (by the Querysub, on received time)
-            - This is important in competitive games to prevent things from undoing a lot of other state, just because one client is lagging badly.
-                - TWO settings, one on a global level, and another on a path level, inside of FunctionRunner (using the writeTime as the base)
-            - MAYBE we actually limit read times based on the writeTime, which we can use to estimate the lag. Ex, they can only read back as far as 4X their current write lag? The default won't be to do this check, but... for competitive situations we do want to do it...
-        - The writeTime will still be the client writeTime, so by default we preserve client side-effect ordering (ex, cast "shield", then "fire", so the shield protects against the fire)
-
-    ++ Fixes nodes jumping around, as well as making it hard to hit laggy nodes
-
-    == Add a check so that dead nodes can't shoot anyone (which we might already have as check?)
-    == Laggy clients will be hard to kill, because by the time they know they are dead, they will have killed their attacker in the past, so they won't be dead anymore!
-
-    Ignored client write times?
-        - With support of "past time reads", we can always change the writeTime (somewhat, we will have to add a special case where all reads are now forced to be past time), without causing a rejection.
-        - This would need to be down on a function level (instead of on a path value)
-        - This would allowing resurrecting bad values, as well as changing the order of effects (ex, shield, then cast fire. BUT, if shield is run at server write time, it could happen after the fire, and so the fire could affect you even though you KNOW the order was correct).
-            - Which... is fine, we just have to make it clear that this reorders the function implications
-            - ALSO, make it clear that this can cause REALLY bad rejections. So if this is done with movement... the client better have really good rejection handling, or it will jitter like crazy
-            - ALSO, if the time of our write is important (ex, if we store keys, and then your position is your timeDelta * velocity), then this breaks that (so it isn't just order sensitive, but time sensitive values it impacts)
-    Expose writeTime to functions
-        - For example, so they can give some leeway in deaths:
-            `if (isDead && isDead < writeTime - 50ms) return "deadCantShoot"`
-
-    ++ Fixes making it hard to kill laggy nodes (by making them shoot at the serverTime, not the clientTime, so they can't surprise clients with "you're dead")
-
-    == Add "gravity" power, which either accelerates all users towards or away from the user
-    == ACTUALLY, I think the problem will be drift. If we are constantly writing, and assuming our prediction are correct, and having them read the previous predictions, we can get really far off from the server
-
-    Client cascading call resimulation
-        - To prevent drift, we can rerun client calls after a rejection. This is a bit annoying, and requires some way to hook directly into rejections (which... actually isn't that hard?)
-            - Basically, instead of clobbering all ReadLocks on call predictions, we identify any ReadLocks on other calls, and then watch them explicitly for rejections. If they reject, we re-simulate our call, writing in such a way that we force our previous call to be rejected (which then triggers ourself for future calls, etc, etc).
-                - We might need to write to versions MUCH farther in the past, as now we need to increment the version, so... yeah...
-        - ONLY if we don't have our result, of course, otherwise we would have no reason to run...
-        - Probably do this in a batch, which considers all calls after rejections, then deep compares values, then follows the call chain to get all candidates for rerunning (and the order they should rerun)
-            - We then need to wrote the original PathValues per result, which... is fine.
-        - ALWAYS update all the way to the latest, OR, doesn't update.
-            - Add a kind of throttling so we don't always update to the latest, to prevent lag causing huge amounts of cpu work. This will cause large "snaps" when we do resimulate, but... it will prevent the browser from locking up for too long?
-                - Set a constant factor (maybe... 60? as 60 * 16 is close to 1000?), of our extra call rate. Then it is relatively easy to throttle, basically, every resolved call value (they are all rejections) adds +60, and every rerun (each value different causes reruns, but can cause many, if there are multiple calls that depend on it) takes away 1 per call.
-            -Make sure to measure this time too, as it will be interesting to see how much time it takes
-        
-    ++ Fixes drift
-
-    == Will there be a "jump" when a user stops toggling their power? If we are only looking at their keystrokes in the past, our predictions will always be off, but it should be relatively smooth?
-
-    == Add collisions
-    == After having very few users this becomes intractable, due to the inefficiencies of calculating collisions
-
-    PHYSICS HELPER
-        - Takes many path+shape+times as input to seed the data
-        - Expose a function which takes a shape, and a range of times (as we don't know which path we are reading, so we don't know which past time to use), and gives all paths which might be close to it
-        - We then take the paths that MIGHT intersect, and do full collision checks
-            - We can do it with "past time" reads or not
-                - "past time" reads allows object to phase through each other
-                - regular reads causes more rejections, which can cause more server load, and clientside jitter
-                    - We should make it so one half of the world does regular reads, and the other does "past reads", and see the differences
-                        - Well, the difference should be, the "past time" world has no backtracking, so if you do a jump puzzle on your end, you did it. BUT, the "past time" world will also have more objects phasing through each other, and you will see other clients jumping on invisible platforms, etc, etc.
-                        - The "regular time" world will be more consistent, but sometimes you will swear you did something (jumping on a platform), but then you will backtrack and fall through it, because it WAS broken before you could jump on it.
-        *** Implement Local cache helper ***
-            - If we create a watchFunction with proxyWatcher, we can make the output synchronously accessible to other functions, as a way to allow unsafe reads
-            - If anything accesses this AND the accessed watcher has never been fully up to date (if it has been, but now isn't... allow it, I guess?), then the accessor is not fully up to date as well. It doesn't tracks it as a dependency, but instead as a "watch"
-                - AND, when the cache is up to date, the watcher will rerun
-                - AND, if the server does the same thing, we will USUALLY result in similar values, BUT, when we don't the server only runs the function once, so it will decide the final result.
-            - For things that matter, such as shooting a player, we will want actually dependencies, but this can be used to limit the things we access which we have to depend on
-            - Can lazily run, and just uses the database, so it will run in isolation fairly easily
-                - When accessing the result, sometimes... register the accessed values as being used (so they aren't unsynced). This might be annoying, and require some batching / delay, but... it is very important!
-
-    ++ Makes system a lot faster, by reducing rejections that don't change the value, and by increasing the efficiency of comparisons
-
-Automatic large value storage separation
-    - If a value is > ~10MB, when we write it to archives, we should ALWAYS break it into another file
-        - Have this decided via a flag on the PathValue, which can be dynamically set via size, or explicitly set.
-    - We should lazily read these values
-    - Garbage collection them after a while, when the file referencing them disappears
-    - We should create a cache of these on disk as well, so even if we do need to read the values, we can often just risk off of the local disk
-    - This will be annoying, but... if we already have disk value offloading, it shouldn't be so bad... AND, could really reduce startup time
-
-Archives disk cache value separation
-    - If we already support large value separation, we can also separate values from archive value disk cache
-    - STILL via some limit
-    - Because it is on our own disk, we can seek inside the files, reading JUST the part we want, so the limit can be much lower, probably 1KB?
-    - This can reduce startup time even further
-
-Network visualization
-    - NONE of this will use our PathValue system, and will instead use specialized instrumentation functions
-        - Will still be realtime though, and show paths, etc
-    - The nodes, and paths, in the browser, with information on time alive, etc
-    - Traffic information
-
-Multi domain support
-    NOTE: Special case LOCAL_DOMAIN, so that we: 1) don't trust anyone else's LOCAL_DOMAIN, and 2) assume no one else trusts our LOCAL_DOMAIN
-    - If you call a function on a cross domain, it won't execute inline, but instead result in a write
-    - rootCertDomain needs to be removed and updated with a dynamic domain
-    - NetworkTrustController needs to maintain trust per domain
-        - Only allow readLocks on paths in trusted domains (otherwise throw, not even committing the write).
-    - Writes to domains that don't trust us readLocks sanitized, so they don't depend on any of our domain values, otherwise our writes will become rejected immediately
-    - Ensure we are still ignoring writes from nodes that aren't the authority on them (I think we are, but this becomes even more important, as now we might partially trust a node, accepting some values, but not others)?
-
-Querysub bootstrapper + git repo file server + local file system server
-    - Basically... instead of requiring a static file server, we can have Querysub automatically host the files / repos based on some data written during deploy (probably specified via a function that is called in something the deploy.ts imports.)
-    - Querysub won't even know which servers it is hosting, and just host a generic bootstrapper, which when run on the client will just do a regular data access with the domain + path to get the entry point
-    - Querysub will basically just have a utility to go from gitrepo => file WITH, load development file system support.
-
-
-Automatic schema usage/generation
-    - Parse the types to generate the schema, then automatically update accesses to use the schema object
-
-PathValue.value disk cache
-    - If an entire server (PathValue server and Querysub) NEVER uses path values, we can move PathValue.values onto the disk, only  reading them off disk when we send values over the wire
-    - This will have to be done explicitly, via a setting in AuthorityPathValueStorage
-    - The values will be read off disk by our serializer, which will ask AuthorityPathValueStorage to read all the real values off disk
-    - The FunctionRunner SHOULDN'T need this, as all of the values it stores should have been used semi-recently...
-        - Actually, only really only the PathValue server requires this, as otherwise we discard values after ClientWatcher.WATCH_STICK_TIME, which is presently 10 seconds!!!??? But even at a few minutes, that value won't be too low. What are we going to do, render 2GB of images at once? (and if it is video we need to stream it anyways, likely from another storage source, as there is no reason to use PathValues for videos...)
-
-Optimizations
-    - Client storage of values, allowing the server to only send Time for large value the client doesn't have
-        - The client can just skip adding the PathValue, BUT, keep track of the last thing that is waiting to add. Then if the value is superceded anyway, it just won't add the full value, after it receives it.
-        - If the values are VERY large the server might be storing them on disk, so this allows the server to avoid even reading them off disk, which is extra efficient.
-    - One time subscriptions + polling subscriptions
-        - For large data that changes frequently this might be better?
-    - Sending very large values seems to cause something to block for a really long time. Not sure what, but... sending 100K writes with writeOnly, and dontStorePredictions, locks up for minutes at a time, then allows a lot of values to get through, then locks up?
-        - 10K writes is a lot better, and 1K writes is really fast.
-        - Maybe... writes on the websocket are buffered, and then the socket disconnects due to having too much send at once, and only reconnects later due to some random polling?
-        - MAYBE we just need better server enforced throttling?
-        - OR, if the problem are that the funtctions are getting rejected (which I am 99% is the problem), we should support "updateWriteTimeIfNeeded", and see if that fixes it?
-        - AND, we also need to handle streaming the updates better in FunctionRunner. It is kind of just batching everything, when it needs to wait (maybe with an "io" delay), to let the outputs clear?
-    - Delta supporting watcher
-        - Take all places that iterate over all paths, and have them work on deltas
-        - The callback will still have to process the changed paths directly (although it will have access to any state), and indicate the deltas to the watches
-        - This could be used in FunctionRunner to make it more efficient
-    - If we track when a path was created, wait longer to archive (2.5 times MAX_CHANGE_AGE), and then detect that is was deleted < MAX_CHANGE_AGE * 0.5 after it was created... we can avoid archiving it. But... then we need to clean up the tracking of when it was created, and that's a whole thing, so... it's just difficult...
-    - The server should batch forwardWrites when it receives them. That way if it is lagging due to a slow (synchronous) operation, a lot of values will buffer up on the network (but the batch queue will be empty, as we will have synchronously emptied it), and then when we finish the request we will have the batch time to gather up all the network requests (which should be a lot! as we can handle a lot of bytes in even just 10ms!).
-        - Of course, this adds ANOTHER 10ms delay, but... that should be fine, we won't get forwarded values often...
-    - We don't need to send back predicted values, when the prediction is accepted
-    - Client side we should only trigger watchers when a watched values actually changes (not just when there are any changes on the watches paths, at any point in history)
-    - Client syncing thrashing
-        - When we no longer need a value, we should stop immediately clearing it, and just mark it as unneeded, and then clear and unsync it a bit later on. This will take more memory, but allow navigation to be quite a bit faster
-    - Optimize parent accesses to not do brute force `.startsWith` searches, and instead maintain some kind of a lookup?
-        - Maybe... although, is this even much faster? It will certainly slow down writes.
-    - Optimize readLocks
-        - Each write stores readLocks very redundantly, making each function have `WRITES * READS` readLocks, which takes a lot to store, and requires a lot to synchronize.
-        - We can probably store a lockHash, to quickly detect duplicates
-        - Wire calls can use lockHash to depopulate duplicates, then re-populate them on the otherside.
-            - Such as commitWrites/forwardWrites, etc
-        - Our storage can depopulate duplicates when storing as well, maybe even storing readLocks in another file?
-    - Clear old pathValueProxy cache
-    - Better object comparison in certain values when we compare PathValue.value values? (only really clientside, I think?)
-        - As well as more object comparison checks, to not notify if the value is set to an identical value.
-    - Streaming / waiting during storage operations
-        - For example, we can stringify the file in chunks, instead of all at once, that way we don't block for > 100ms. We should wait ~10ms, giving a lot of requests time to clear.
-        - This isn't going to be especially big for large datasets, which may require merging everything at once, which with our current algorithm could take seconds!
-    - More efficient wire size and serialization speed
-        - If we write to 200K number values, which all depend on the previous states of 200K values, we end up sending 94MB, which takes 1200ms to serialize. We also receive a lot of data back for all the valid states. It might not be easy, but... it would be nice to optimize this, in some way.
-        - Probably via schema handling?
-        - ALSO, 94MB exceeds our default packet limit, so it doesn't even send! We could fix this with compression, but then it would take even longer to serialize!
-
-Forward retry code
-    - We should maybe retry to forward writes if we can't forward them (and we aren't an authority)?
-
-Wait a bit before loading values, in case a server wrote values, died, but the remote storage system isn't serving the read back yet?
-
-Changing authorities bug
-    - If an authority changes, it is possible for it to receive values it doesn't care about, and then respond saying the results are valid. However, they aren't, instead it just ignored the values.
-    - ALSO, it is possible we will send a client ends up watching valid states from multiple authorities, due to being unsure if any are down, etc.
-        - AND, we could end up subscribed to multiple authorities for a single path (due to watchValueLocks), which will result in thrashing.
-
-Prediction limiting / server overload protection
-    - If the PathValue server gets too far behind, we should stop predicting values, and start waiting for updates until the server can catch up
-    - This can also help when the server goes down, OR, more likely (hopefully), when the client's internet goes down.
-    - ALSO, the first thing to try is really to try switching servers, which will give us a form of automatic load balancing
-        - Although... if the limiting factor is our internet... then switching servers will actually make it worse. So... idk... this is actually difficult!
-        - MAYBE the server has to ask us to switch servers, and all we do when we find slowness is to slow down our writes?
-
-Clientside function threading
-    - Clientside it would be fantastic, as it would allow render functions to NOT block the render thread. This means we could run multiple threads, keeping the application responsive even if a very slow component is rendering.
-        - We can convert dom event callbacks (or any closed values) to { functionContents, closedValues: { [variableName: string]: unknown } }. If the variable values are either serializable, OR, can be linked to exports (ex, misc_1.isEmpty), then we can recreate an identical function just by using eval.
-            - If we do this, we would we ALWAYS want to recreate the function. This ensures there is consistency if we call the callback cross thread, or on the same thread.
-            - We can store this style inside of the synced state, and make it available to be called directly.
-
-Serverside function threading
-    - Required to kill very slow running functions
-        - They will of course we able to change their default timeout time
-        - They will need to be given an error result, to prevent them from rerunning
-
-Serverside rendering
-    - Create a mounter which can go from vnode => string
-    - Clientside, probably render from scratch, and just have our clientside mounter check for existing DOM nodes (probably just when instructed to, which will happen on initial render, maybe... always on initial render?)
-        - Events COULD be an issue, or... we could just assume any unknown nodes have no event handlers?
-        - I THINK we can probably ignore events before we render, at least for version 1. The app is mostly a SPA anyways, so the time before render should be VERY short, and then everything should involve re-renders and NOT re-navigations. For version 2 we can capture everything globally via a built in script tag, and then re-trigger them once we actually mount (and either block, or delay links, so we don't get in a state of constantly trying to load/render, getting 50% of the way there, but then the user navigates via a link).
-
-TESTS
-    - Tests which track the code used so we know which tests need to be rerun
-    - Render tests, with render JSX and paint it
-        - Can use a partial JSX => paint cache, so most of the time if the output doesn't change, OR, if the output doesn't change enough to change how it is painted, we don't have to repaint it.
-        - Also, a test harness that can click on vnodes and simulate their changes would be incredibly useful!
-            - This would get rid of the chrome javascript runtime step. We might still use puppeteer, but also... it is possible we could just use blink, or some other library, to just go from HTML => image extremely fast (realistically most pages should take < 50ms to render)
-    - True test coverage checking
-        - NOOP expressions (x + y becomes 0, etc, a function call that returns a number becomes 0, etc, etc), and see if any test changes. If not... then that line does not have test coverage
-
-=== Development tools feature complete | TASKS = 22 ===
-
-Latency / load weighted node selection
-    - When they are multiple candidates, prefer the closer nodes
-    - ALSO, prefer nodes that have less load, allowing a form of automatic network balancing
-    - The ability to switch nodes would be nice as well
-        - Switching PathValueController nodes should be easy, we just subscribe everything to a new node, then disconnect from the previous node.
-        - We switch Querysub easily (using multiple), and disconnecting from the previous ones as soon as all calls on them have finished running (really that is true for most nodes, we don't want to disconnect when we have pending calls!)
-
-Speculative function calls (calling functions that haven't been requested yet, just to sync the state they might need)
-    - Will be a bit involved, and might even require some clientside cooperation (ex, to detect which actions are possibly via which event handlers are being used), but... could result in a HUGE speed improvement
-    - Will have to determine what functions are most likely to be called
-        - We will likely need to give some context information for functions, or at least know their index, and maybe even page position? And maybe even the mouse position?
-        - If we run functions calls when the user moves their mouse towards a button we can probably increase our estimate quality with little effort
-            - We should test to see how long the mouse dwells over a button before clicking. Even 10ms would be useful. And also the average mouse speed. If we find users usually click after dwelling for say... 10ms, but take 20ms to click, we can get a 10ms headstart on the call!
-    - Will re-render after the click and then pre-sync the new reads needed
-        - If we do this on another thread, and somehow set the new syncs with low priority... this should be basically free (as if the user is moving their mouse around it isn't a background task, and we can at least monopolize a single core!)
-    - Will also pre-populate the clientside function cache, so when they really click it we can get the prediction very fast
-
-Delta based render cache
-    - The best idea is to mark reads as being delta allowed, and THEN, marking output values as being reduceable. Then we just run with partial values given to the read, and merge the new writes with the old writes. IF the writes differ in any places that are not reducable... then we have to run with the entire set of data (and it results in a warning). But if not... it just works!
-        - The reduce functions can be 
-    We already know the delta when we re-run watchers (as we reran them for a reason!), 
-    - Maybe at the component level? Although... I kind of want to implement it for just arbitrary data
-    - A function will have the ability to read from an object/list with a "delta" read
-        - It will still receive the object/list, it might just partial
-        - EXCEPT, we might need to return special state for deletions?
-    - If the only reads of the object/list are delta, then the next time there is a change, only the new/changed values are provided
-    - The output is analyzed, and the differences between the new output are found
-    - For every different write, we determine the update technique
-        - This will probably involve setting some special value on the write?
-        - Examples would be "sum" or "join" or "insertSorted"
-        - The most generic case receives the previous state (that has changed, already drilled down), and the new state, and updates the previous state (in memory) so it now has the new state as well.
-        - We need to handle writes that propagate deletions
-
-Blocking FunctionRunner call mode
-    - Will probably be slower in most cases though, as each value accesses required a round trip with an authority to read the value
-        - Might be useful for code with expensive side-effects?
-    - Add a BLOCKING mode to PathValueProxyWatcher, which uses Atomics.wait to block until a read value is synced.
-    - This potentially allows for efficient function evaluation, by not requiring any code to rerun.
-    - This is only possible if something can mark a value as synced on another thread, and absolutely not if we are single threaded (Atomics.wait wouldn't even work on the main thread of a single thread app anyways)
-
-Function results!
-    - Specially marked "specializedHardwareCalls" functions will be able to "return" a result
-    - Only possible with blocking FunctionRunner mode, as for the functions we want to return results we also only want to run them once (for expensive operations)
-        - Of course, this doesn't prevent: 1) slowness cascasding syncing, or 2) result invalidation
-    - Hmm... we also want the ability to cancel calls?
-        - Maybe calls can do this without themselves?
-    - Hmm... if the calls are so slow... maybe the user should just manage a queue themselves, so they can see what is going on? It really isn't that hard, and if they have to manage special hardware anyway... then it won't like this would prevent them from just quickly scaling a function?
-    - At this point we should have support for unsafe function writes. We MIGHT want to make the writes from this unsafe by default?
-    - By default probably allow write time adjustment, to prevent invalidation (at least when we start the function call)
-    - The return object will be { type: Symbol("hardwareCall"), call: number, path: ".,querysub.com,.test" }
-        - ALSO, unioned with a type for an error { type: "error", errorMessage: string; errorMessageFull: string; }?
-    - The function return types will be modified so any callers have to handle this symbol type
-    - The proxy watcher will temporarily register to receive all calls, so it can call them
-    - The proxy watcher will search the output writes for these values
-        - Only shallowly, which is fine for now, as if they nest it I believe the Symbol will cause a throw, so the wrong value won't be written and not noticed
-    - The proxy watcher will then actually construct the call objects, telling them where to place their results
-    - When the FunctionRunner evaluates the calls, it will place the results in the specified paths
-
-Support VERY large data sets (> local disk size)
-    - If an authority hasn't been used for a while, dump the memory for it, and... become "unready". Then, if someone accesses it, go about loading it again.
-    - This only works if we are sharded, but there isn't really a better way.
-    - We should test this with some very large values.
-
-Automatic sharding of ValuePath nodes
-    - For KVP nodes as well as Function nodes
-        - More important for function nodes, but... it could help for KVP nodes as well?
-    - IF we have high contention, and many servers, the resolving process takes SERVER_COUNT * SERVER_LATENCY, which could be high. Automatic balancing can help fix this, by finding a way to identify overlapping writes via reads (ex, spatially hash read paths, and if those hashes are ===, see if the spatial hash of the write paths are as well).
-        - This reduces the SERVER_COUNT involved in contention, which... fixes the issue!
-    - We will need to support remotePathAuthority to get this to work
-    - Needs to be somewhat generic, so we can use it for both PathValueController, and FunctionRunnerController
-    - Nodes will inform their parent path nodes of their utilization, so parent nodes can redistribute child nodes accordingly
-    - Oversaturated nodes will split, pushing newer nodes down to more specific traffic
-        - Ex, from ["x"] to ["x", "x.[0 <= hash < 0.5]"], ["x.[0.5 <= hash < 1]"]. AND, when a node has enough descendants, it will stop with all child paths, and the parent will just be ["x"]
-        - BUT, we should only shard when needed. Otherwise syncing parents can become overly expensive? (Ex, if we have 100 servers, we really wouldn't want to split a small collection across 100 servers, as then it would take a huge amount of work to sync the keys, and if there are < 100 keys, it would be needlessly slow)
-    - Nodes who have parents that have insufficient members will remerge with their parent (perhaps completely taking over the parent, if the parent completely disappears)
-    - Nodes with child nodes with insufficient members will redistribute, or remerge into one path, as needed
-    - Test on a local machine, spawning many processes, killing them, etc
-    - We need to fix our path limitation thing
-        - Detect the size limit of our underlying file system, and use that
-        - This allows us to turn on long paths on windows
-            - WARN if long paths are not enabled
-        - Use this both to constrain balancing AND to have archives write to the most specific path possible, within the current limits
-
-Programmatic digital ocean droplet launching
-    - We will want some kind of administration page that the app serves by default, requiring NetworkTrust and localhost to access.
-    - From this page we should be able to:
-        - import digital ocean api keys
-        - see all currently configured droplets
-        - launch new droplets
-    - All launched droplets will be automatically configured with the code, given trust, and configured to auto restart all the necessary process, etc (so they should be able to run unattended after starting)
-        - This means we can get rid of the emails to add trust, as nodes will no longer just "appear", but instead by instructed to appear, at which point we can explicitly ask for their public key and trust it (we should never provide the key, as this is incompatible with a transition to hardware keys).
-    - We also want the ability to monitor traffic, and make recommendations about having less/more droplets
-        - These recommendation should display in the UI
-        - There should be daily or weekly emails about recommendations to some admin email account.
-        - EVENTUALLY we might have these recommendations automatically followed, for scaling, although maybe only to certain degrees, as automatic scaling can be dangerous.
-    - AND, it would be nice if we could view/administer MULTIPLE domains at once, OR AT LEAST summarize traffic recommendations for multiple domains at once.
-
-Automatic digital ocean droplet launching
-
-History for the purposes of reverting data
-    - EVEN just backups works, but... storing a transaction log is preferred
-    - If we store writes, we can re-simulate to a previous point in time. It might be slow, but it is better than losing all your data due to a database drop.
-    - If we store clobbered data as well, then we will be able to restore even if our history size exceeds the max and needs to be truncated
-    - Kind of required if we want to use it to store any important data...
-
-Fix hardcoded email and TOS accept in certAuthority.ts
-
-AST parsing to detect global side-effects in functions
-    - Global side-effects are going to break things. We should at least mark root functions that do this, OR, that have any child calls that do this.
-        - Who knows what libraries we call are going to be doing, they could have all kinds of unintended side effects involving the disk and who knows what else!
-    - The strictest mode would be the disallow global side-effects unless it is enabled per function call
-    - ALSO, AST parsing to detect module side-effects (that impact values outside of that module)
-
-Write proxy replacing
-    - If we have a lot of local writes it can get slow, due to the proxy. If we replace the writes with function calls it could allow a lot more local writes.
-
-Schemaed proxy replacing accesses
-    - Instead of passing an array, find common accesses and replace with a call to a factory function to create a schema, which generates a function, which we then pass the remaining variables to
-        - Try to put all of these beside the databaseTyped (or at least at a module level!), so we can view the code without source maps and see all the examples of accesses. Their will be 1 for each line that reads from the database (that we can find)
-    - If we set this schema in the write itself we can greatly reduce write overhead (especially if we have keys that are numbers).
-    - We can send a binary format to forwardWrites and onValues
-    - Maybe just for the wire communication. But sending a buffer as the payload, instead of a value.
-
-Fix multi authority writes
-    - Multi authority writes are annoying, because it causes issues with predictions. As in, you write, predict your value, but then have to wait because one authority is taking a long time to resolve. THEN, another write uses our prediction, but this time only uses the fast authority. This results in the fast authority rejecting our write, until we can send it the full writes, which may take a while if the slow authority isn't resolving!
-        - AND, if the slow authority takes a REALLY long time, we might never commit the write, resulting in a valid state, BUT, we can no longer commit any writes! Although, maybe this state is okay, as if we can't write the full write... we probably shouldn't write?
-        - And... maybe this whole issue is fine, as the authorities will eventually resolve, and then it will be fine...
-
-=== Shippable features complete | TASKS = 12 ===
-
-Triggers / data pipeline / autorun
-    - Basically an abstraction of what FunctionRunner does (where it watches the database for values, and then writes other values to the database)
-    - Setup via a value in the database
-        - Like a function write, but, this function reruns every time anything it read changes
-    - Will keep the same thread (maybe a dedicate thread?) for a long period of time
-    - The ability to watch data, calling a non-committed function when it changes
-    - The function that is triggered can call synced functions, committing values
-    - Will be able to sync a lot of state, and also keep values in untracked state (if it wants to)
-    ALTERNATIVE
-        - Just use FunctionRunner, and if you need to use dedicated hardware, use another domain
-
-1.3) Buffer support in SocketFunction? Hmm... I mean, if we have GB sized files, that would required... And sending a 20GB video file is maybe feasible? Hmm... or... maybe not? Maybe it is more useful for 1MB chunks, which we don't want to have to decode each time? So this is really just "increase data throughput"?
-    - Essentially, we want a JSON.stringify that converts to a Buffer, and also supports Buffers.
-    - We will probably eventually support threads, so... maybe we just use Accessors2? Hmm... Or at least some rework of Accessors2?
-    - Would also need to support it in function argument serialization / deserialization
-
-Error notification system
-    - Some kind of immediate notification + email digest system
-
-Runtime FunctionController type checking
-    - We should check the type of all arguments against their typescript type by default. That way accidental prototype contamination is less likely. And maybe there are other security vulnerabilities?
-
-Backups
-    - Taking our entire database, and backing it up periodically
-    - We can probably check the time in the last backup, and use that to infer what values will definitely be in it, using this to backup less data (nothing that is already backed up), making this efficient (to write, not necessarily to read)
-        - Once in a while we would want to write the full state, otherwise after a few thousand backups restoring could take forever.
-        - We can also be somewhat efficient about reading from the database files, using block create time to know if they are only old and already backed up data.
-    - At this point we will already have a management interface, so... this should be configurable via that
-        - We will have to think about how to rate limit it. A lot of databases would probably work by just streaming every single write, then compressing them after they exceed a certain total size.
-            - AND, when we compress, we probably want to default to the "compress old data more", so we have logarithmic seeking capabilities.
-    - Might be made redundant by the history navigation feature?
-
-=== Feature complete | TASKS = 3 ===
-
-Path queries for synchronization
-    - We could support a way to query paths (likely via a parent selector), to do synchronizations. It makes it slower for the server to continue to send values (each new value required evaluating the selector), but... it could make selecting large amounts of data (millions of values) much faster?
-
-Local network call forwarding / neighbor prediction
-    - If there is a write on the local network we can get faster then waiting for it go to the server or back (by local network it mostly means on the same machine, but different process / browser tab / thread).
-    - We will need to ingest these values as regular predictions, except because we aren't writing them we also need to watch for their valid states
-        - We also aren't getting if the write is even commited or not (which tells us if it was EVER valid), so... we basically need to assume the value is invalid if (AFTER the valid watch is done), we don't get a valid notification after a short period of time
-    - We also need to validate writes coming from neighbors, to ensure they are recent enough, and maybe we should do some other checks?
-    - Having this work for function writes would be nice, but... we REALLY have to trust the neighbor for this to work?
-        - Well... we could always run the permissions checks? In which case... I guess it would be safe?
-        - Oh, well.. maybe permissions checks aren't that important. If they are on the same network they are sitting in the same room, so they are somewhat trusted!!!
-            (And worst case they just cause the data to temporarily change, which isn't the end of the world...)
-        - AND, we can have neighbor prediction off by default, and only turned on for certain application (maybe on by default in the browser).
-
-Function call syncing (as opposed to PathValue syncing)
-    - If we have a physics based system where we predict the physics loop every tick, we can predict a HUGE amount of values (millions), possibly even using the GPU. HOWEVER, the server won't understand our prediction efficiently, and so it will need to send as all the values anyways!
-    - If instead of syncing the paths we use, we sync the functions that impact these paths, we can have a massive state be kept up to date without saturating the network
-    - We will have to be told the hash results of these functions, to ensure we are calculating them correctly.
-        - So, a server instance will have to basically watch all the paths itself, to figure out which functions apply.
-            - And... presumably the results need to be committed anyways, otherwise this is just a different database! Although... if the function runner and KVP owner are on the same machine, the transfer of paths should be somewhat efficient? And only snapshots need to be saved, so... this could work?
-            - Basically, you would run the KVP node, and the function runner, on the same machine.
-            - Some kind of pipeline would trigger the physics function to tick every frame
-            - This pipeline would be also predicted on the clientside, so it can consistent know about ticks
-            - Eventually the client will know what functions were run (it should be pretty accurate at knowing this!), And the hash results. The hash results can come later, as the client will know that it will have to simulate old physics results a few times as course changes trickle in.
-
-The ability to mark writes as "no persistent storage"
-    - Useful for things such as streaming video, which can then have a pipeline that combined it and stores it in B2 (the pipeline would mark what is stored in B2 and the urls for that content, and the rest of the data would be available on the network directly)
-        - This makes live streaming possible, but also accessing terabytes of historical data.
-        - This is different than just video uploading, which should go directly to B2, because it trickles, requiring something to aggregate it, while also requiring instant access.
-
-Read function triggers
-    - Allowing setting up a value that calls a function every time it is read, to produce the data (and non client predictable function)
-    - Allows for exposing an off-network source (like a database) to the network
-    - By default will just mark everything as valid, BUT, we could add valid ranges
-        - For databases like mongodb we could roughly do it via changestreams, to warn if the value has changed
-        - For blockchains we can perfectly detect if the value has changed
-    - For the writing to external sources side... we would need "unsafe" functions, that don't add any readLocks (so they can never become rejected). Because we can't reject values in external sources!
-
-FunctionController path syncing
-    - For small number of reads, the latency of the read is acceptable for the trade off of reduced complexity of FunctionController.
-    - HOWEVER, for functions with a HUGE amount of reads (GBs of data), we will want to sync it instead...
-        - We still need to run historical functions, so... we basically need to:
-            1) Ask for ALL future writes on a path
-            2) Ask for all writes on that path (up to what the authority stores, which is up to and including the last golden value, which should be enough history)
-            3) Sync the valid state of all writes we receive, forever
-        - The hard part, is trying to keep the paths synced simple. If we just sync every access... then we could be syncing tens of millions of paths... which means not only does an authority have to store that much data, it also has to store watchers for all of it... which... will probably be slow?
-            - Although... maybe not... it is just a Map<string, string>?
-
-Support arrays, via adding another transparent property
-    - Basically the object properties, BUT, it will specify it is an array
-    - The proxy will have to be told of this, and... function properly?
-        - I think if it's object is [] it will work? Maybe? We should double check how proxys handle arrays...
-        - Does push just map to a set plus a .length update?
-        - And what about splices!
-    - This will be annoying, and probably slow...
-
-Actually Object.keys() ACID handling
-    - Ugh... very annoying, and difficult
-    - A path authority can sync child keys for a range of time (similar to how it syncs a path for a range of time), and then compute a hash of them (sort and sha256). It won't be efficient, but... it should work.
-    - Will be OPT-IN, because the slowness of hashing the keys is almost never worth the ACID nature of it
-
-Object.keyRanges()
-    - If we use search patterns that line up with our range sharding, we can efficiently access ranges of keys
-    - We will probably need to add more details in our range sharding
-    - We might want to specify sharding characteristics via some callback near proxyWatcher.databaseTyped
-        - Ex, schema.dataSharding(database => [{ path: database.logs, map: key => +key }]
-        - Ex, schema.dataSharding(database => [{ path: database.maps, map: key => key.split("_")[0] }]
-
-Old data cleanup ability
-    - Some way to actually delete old data
-        - Maybe via both setting all values to undefined, and having nested undefined no longer stored
-
-Ability to selectively undo history
-    - Probably recursively by path
-        - We would than have to store history under those paths
-    - Probably with a limited history (ex, 30 days?)
-    - The ability to select changes by values / by time, taking everything in the same changeset (via looking at readLocks), and adding a new change that reverts the change (if the revert value is still the latest)
-    - Ex, undo all changes by user for the past day, one specified paths
-
-Disk support
-    Local disk support instead of backblaze
-        - AKA, implementing backblaze using the local disk of many nodes
-    Recent changes disk backup
-        - Store changes that are too young to be archived on disk
-            - CAN'T use archive the values, as those values are assumed to be valid, so... needs a new format, which is just used to store changes, in case of a crash.
-        - If we have a single node, we can stream our changes to the disk, reducing our data loss on crash from ~10 minutes, to < 10 seconds, with very little disk IO
-            - We can have a flush time option, which can be set even lower, which will increase IO, but can reduce losses from crash to any point we want.
-
-C++ (.cpp) import support
-    - Via WASM compilation, which can be done via https://github.com/sliftist/cpp-portable-loader and `clang-wasm`
-
-C# import support / other language child processes
-    - Would only work on the serverside
-    - Would have to run a child process which runs the C# code?
-    - If we are going to run another process, we can just run any easy scriptable language. Python would be easy to get working as well. On a smaller scale, Java would work too.
-        - Ideally we automatically generate typings, but... even if we didn't, that would be fine.
-    - C++ would be useful too! As a lot of the time this is useful for making OS calls, which are easiest in C++.
-
-Rust import support
-    - Compiling Rust to wasm is even easier than C++, so... we could probably do it fairly easily
-        - I'm not sure how to get typings, although they aren't strictly needed
-
-=== Advanced optimizations | TASKS = 2 ===
-
-=== \/ Speculative tasks \/ ===
-
-- function call results
-    - Some way for function calls to return T|undefined, and be given the location they are assigned to, so they can automatically write the result there
-        - Is basically the same as adding the path, so... this is probably a good way to do it.
-        - Can still with with function predict!
-        - Passing the path to the function call is a bit annoying, but... if we return a special value, we can observe where that gets written to in the proxy, so... it isn't so bad...
-            - We can even allow writing to multiple output locations
-        - Really only makes sense if the function is slow. Otherwise, is should just be an inline function!
-            - We should add in function level sharding, so we can exclude function calls from function runners that don't opt into those. That we can split up slow functions from regular functions.
-
-- "limits", to limit resource strain any node can put on PathValueServer
-    - limit the total number of paths watched
-    - limit the number of writes per second
-    - limit the total bytes per second of writes
-    - maybe limit onValue callbacks, depending on size, etc
-    - Maybe dynamic limits, depending on load
-
-
-- Fix our JSON serialization code converting undefined to null inside of arrays
-    - This impacts function calls, which can be really annoying / confusing...
-
-- Actual query subscriptions
-    - Tell the querysub server that function we want to run, and everything it watches will be returned
-        - Anything the server misses can be additionally subscribed to
-    - This allows the cascading latency penalty to be equal to the querysub server latency. WHICH, if it is on the same server as the PathValue server, can be less than 1ms (we will need to turn off some automatic batching to lower it this far)
-        - AND, if the querysub server did a full PathValue sync, asking for all values on all paths... the latency would be 0.
-            - We would need to be able to fit the entire database into memory, but... if we allowed `watchRecursive(path)`, and either shared Querysub, OR, had queries explicitly map to the path they wanted (which hopefully would either be small, or shared with another server), then... it would just be efficient...
-    - AND, the client could even subscribe to `watchRecursive` as well?
-
-Better FunctionRunner startup
-    - We should sync all of the repos we will need BEFORE taking any shard space? As the clone could easily take minutes, which would cause functions to timeout.
-        - Although... maybe just having it better distributed would solve that problem, as other nodes should automatically pick up the slack?
-
-CORS support, so querysub can be accessed by other domains
-    - We would read the CORS values from the database, and they would be set in, or similar to "yarn shard-deploy" / "yarn deploy"
-    - Only querysub needs CORS, as the browser should know to only access it
-
-Serverside "local" writes
-    - Update code to special case "local" writes, so that we never sanitize remote writes so they never use ReadLocks on "local" writes.
-        - BUT, allow "local" writes to have ReadLocks on remote writes (IF we are trusted on that domain). Our system should automatically subscribe to valid states, and maintain the valid state of "local" writes. This prevents our "local" values from having invalid values that stick around (as in, in a cache), even though we might end up writing them to a "remote", which would make them invalid.
-
-Have deploy log the % files and % lines that are NOT allowclient
-    - This determines which files we might serve clientside
-    - We only really care about files NOT in node_modules, but, in our folder?
-        - Although, maybe we should consider any private npm packages as well?
-
-Lazy Function / Function Prioritization
-    - If we have functions that can run at any time (but must run in order), then... we can run them later, when the server is more free.
-        - AND, we can even go as far as to detect when the predicted outputs of them are needed, running the functions at those times?
-    - Maybe we would actually want this to be functions triggered one read, but only triggered once?
-
-Ability to disable syncing of paths
-    - Possibly depending on context, such as page
-    - Also, possibly just to reduce the update rate for certain paths
-    - This fixes the issue of lag (either network, client, or server), when many pages use a commonly changed value
-        - Ideally we WOULD like it to be updated, but... if we don't need it updated, and 99% of our server capacity is spent updating a few values we don't care about...
-    - We want this to be able to be added post-hoc, potentially live (at a data level, not a code level), 
-    so the site can be quickly optimized on the fly
-        - But of course, the code will also need the ability to specify it doesn't want updates
-    - Will basically function as a watchLatestOnce, where it gets the latest value, and then no more.
-
-Better undefined support
-    - We COULD make it so for the last run of a function, after all values are synced, we run again returning undefined for every value that is fully synchronized with no child values synchronized.
-        - If the first run has every value synchronized we would have to run again.
-    Pros
-        - This MIGHT make function run more naturally
-    Cons    
-        - Functions are more complex to run (and must run more times)
-    Alternatives
-        a) Just use atomicObjectRead if you want to read undefined
-        b) Use `+x`, `x + ""`, OR `${x}`, to coerce it to a primitive
-
-Support instances in our data?
-    - Benefits
-        - Instead of passing around string ids everywhere, and having programmers have to know which functions use it, you can pass around a class and just get autocomplete
-    - Costs
-        - Slow
-        - Harder to debug
-    - Alternatives
-        - Compile time instances => static function + id call
-            - Basically, we can "store" and "pass" around an instance, and even "new" it. But
-                new => create random id, and store data in static location under that id
-                function calls => call static function with instance (which is a string) as a parameter
-                instance accesses (including this.) => lookup data from root with static lookup
-            - This is harder to do, but... faster, and easy to debug.
-        - Function modules with local .databaseTyped call
-            - You can go to defintion and see all the exported functions, which are probably how you mutate the data (especially if the .databaseTyped call isn't exported!)
-    - BASICALLY, everything starts as a static call. But... those calls may wish to create class instances, and store those (ex, binding data and functions)
-    - SO, on seeing a set that is a class (ex, has functions), instead of just stripping the functions, we...
-        - Figure out if it is a deployed function (has exposedRootClass decorator).
-            - If not, we don't store it
-        - Store a new InstanceId in our schema with the data provided
-            [DomainName][ClassName].Instances[InstanceId].Data = Data
-        - Replace the write with this new instanceId
-    - On accessing a value that... looks like an instanceId? We...
-        - On reading values we use the Instances.Data path instead
-        - On calling functions we load the class from the Source info, and call it
-            - Specifically, providing a this context equal to the data value.
-
-Clientside "local" FunctionRunner
-    - In theory, some operations will heavily depend on some server function call completely successfully. BUT, also store local state?
-    - IF we never need this, then we never need to implement it. But... if we find a case where this is useful, in which just moving the data to the remote is not viable (and there is no alternative)... then... we could do it?
-    1) Allow "local" functions to run using FunctionRunner, so they can automatically rerun when rejected
-        - We will require the ability to mount FunctionRunners under the "local" domain explicitly
-    2) Allow (when specified) "local" functions to depend on "remote" writes
-        - This requires allowing syncing valid state, etc
-        - Without runing FunctionRunner rejections are usually not preferred. But with running it, rejections can result in a rerun, and so are not as bad.
-
-Intercept DB, to allow writes without actual ownership of a domain, cascading to store all values that depend on these changes as well.
-    - Maybe only useful for hacking together application, or development, but... seems like it would be pretty cool?
-
-Function call results (for cross domain calls)
-    - Probably not a good idea. Maybe... just never do this...
-    - Non-cross domain calls are all inline anyways, so this is only for cross domain calls
-    Something like: QuerySub.afterFunction(() => OtherController.queueAndRun(), result => TestController.finished(result)), which tells FunctionRunner to do some extra stuff after a cross domain call finishes.
-    - We COULD ALSO use async/await. We KNOW when the synchronous part of a function finishes
-        - Ex: `let result = await QuerySub.afterFunction(() => OtherController.queueAndRun())`
-        - As we are broken in on the machine, we don't need ALWAYS resume (although if the node is killed, and the function needs to be rerun, we would need to re-evaluate the previous call to catch up).
-        - We would need know when the await returns, wrapping it with some code?
-            - This is... impossible? Although... if the thread is CLEAN, we could just assume the next function will run, resolve, then wait for a promise, then assume the function is finished? It gets REALLY difficult
-    - This will break up the ACID nature of the function, which is fairly unfortunate
-
-Size optimization
-    - Move register calls into a .client.ts file, so browsers only need to import the interface, and not the implementation too!
-    - Better HTTPS endpoint which prebundles some calls, so we can bootstrap faster
-
-Streamlined externals, via a custom site that takes payment and handles setting up all the other services
-
-Lazy connections
-    - NetworkState creates a lot of connections, which have very infrequent traffic. Instead of having N network connections, we should allow some connections to be marked as less important, and closed when not used for a bit. Although... if we say had, 10K open TCP connections, is this a problem? Hmm...
-
-Synchronize time ourself
-    - We can't trust machines to synchronize their time. Synchronize it ourself in NetworkState (OR, maybe just by using NetworkState), and use that instead of Date.now() everywhere!
-    - We'll want to use a waitForTimeToSync() function in places to gate our major code locations to ensure we always have an synchronize time, so we can expose a getTime() function that is synchronous.
-
-Replace ALMOST all external services
-    - We can run our own DNS server
-    - We can replace firestore with our regular storage controller
-
-Very short expiry times on thread certificates, finding some way to automatically update them while running
-- At first only node <=> node
-- BUT THEN, our proxy should be able to handle updating the peer cert, as it can probably call renegotiate?
-- Our system for the client to update it's cert after the fact can likely be reused to trigger an update of credentials?
-    - I assume outstanding connections won't be killed if their certs expire, so... we would actually want to
-        add an additional check to close a connection if it's cert isn't updated? Or... we could just call renegotiate, which
-        I think serverside TLS connections can do?
-
-Public release?
-Get benchmarks on https://benchant.com/ranking/database-ranking
+
+
+Trigger values
+    1) validStateComputer.ingestValuesAndValidStates
+    1.5) authorityStorage.ingestValues
+    2) authorityStorage.ingestValues
+    3) lockWatcher2.watchValueLocks
+    4) validStateComputer.computeValidStates
+    5) lockWatcher2.getValuePathWatchers
+    6) pathWatcher.triggerValuesChanged
+    7) pathWatcher.triggerLatestWatcher
+    8) PathValueControllerBase/clientWatcher.localOnValueCallback
+
+Source of values
+    1) clientWatcher.setValues
+    2) PathValueCommitter.commitValues
+    3) (predictions) validStateComputer.ingestValuesAndValidStates
+    4) (non-predictions) PathValueControllerBase.createValues
+    5) validStateComputer.ingestValuesAndValidStates
+
+Archiving values
+    1) authorityStorage.ingestValues
+    2) pathValueArchives.archiveValues
+
+TODO: pathValueAuthority detection
+
+
+Core parts
+    - Valid evaluation
+        - Needs external value syncing, which can give it all past values, and all future values for a path
+    - Value syncer
+        - Needs someone to give it authorities, and tell it when an authority goes down (at which point it finds a new authority and syncs the new values)
+    - Authority tracker (supports registering, checking, notifying, and generally synchronizing all authority information between all authorities)
+        - No requirements
+    - Authority setuper
+        - Needs authority tracker
+        - Needs value syncer
+    - Authority (stitches together functionality)
+        - Needs authority tracker
+        - Needs authority setuper
+        - Needs value syncer
+        - Needs valid evaluation
+    - Disk value saving/retrieval
+
+FunctionRunner is funny. It wants a full history, for reading, but does not calculate the valid states of PathValues.
+
+
+Optimization / optional features list
+    Valid Evaluation
+        - Only need to sync some data, not all data. 
+        - event: sometimes we discard data
+        - isTransparent: Sometimes, even though two values are different, they can be considered equivalent. 
+        - gc: Older values won't change their valid state, so we can discard their locks.
+    Value Syncer
+        - Path watches
+        - Parent path watches
+        - Authority path (tree) watches
+        - Valid watching (authorities know this, so it's free to also sync this)
+        - Only latest/valid state watches (no history, just latest, more efficient)
+        - Special connection clobber logic (you can't store the history after you disconnect if you are not the authority)
+        - Special logic
+        - Reused for internal watches (rendering, etc)
+    Authority tracker
+        - Immutable authorities, to simplify usages
+        - Path requests => authorities might take slices from multiple authorities
+    Authority setuper
+        - Once on startup, so we don't change authorities, to simplify implementation
+        - Leverage disk snapshots to make loading faster and lighter for existing nodes
+    Authority
+        - The stitching together of all the values is reused by the function runner in order to rerun functions when they are invalidated
+    Disk value saving/retrieval
+        - Only old values with discarded locks are written, and the history is compressed occasionally 
+
+
+
+
+
+*** Goal is to get to === Full public website stack milestone === ***
+=== 3 tasks to fully fledged syncing KVP database, 2023 / 25 / 2 ===
+=== 1 tasks to fully fledged syncing KVP database, 2023 / 26 / 2 ===
+=== FINISHED fully fledged syncing KVP database, 2023 / 28 / 1 ===
+=== 6 tasks to "Full public website stack", 2023 / 31 / 1 ===
+=== 5 tasks to "Full public website stack", 2023 / 31 / 1 ===
+=== FunctionRunner bones done!, 2023 / 31 / 2 ===
+=== Disk garbage collection bones done, 2023 / 11 / 3 ===
+=== Access for non-network nodes (via permissions) done, 2023 / 19 / 3 ===
+=== Clientside + clientside function prediction done! 2023 / 23 / 3 ===
+=== Prototype SD app with 2 creation modes, arbitrary model support, paint, diffuse, openpose, onnx model generation, etc. 2023 / 10 / 1 ===
+=== Lots of failed embedding training research. 2023 / 12 / 18 ===
+=== Better single source of truth. 2023 / 12 / 31 ===
+=== Better serialization format. 2023 / 12 / 31 ===
+=== Fixed storage / multi process storage. 2023 / 12 / 31 ===
+
+TIMING: Transactions, currently about ~1.5ms per transaction + 10us per write and read path in the transaction
+    - About 5ms latency, the fast times are only if you don't wait for transactions to finish (because why would you wait when the whole system is based on predicting writes?)
+    - With 1 watcher (with 0 watchers transactions are probably about 0.5ms per)
+    - Much of the transaction over is serialization time, mostly in JSON.stringify (to send the value to the watcher)
+    - MUST FASTER NOW, if they are batched
+
+TIMING: Function calls, about ~1ms-3ms for a trivial function, if it can be batched with similar functions
+    - Slower if we need to sync new paths
+
+
+Fixed function props / function closure parsing
+    - Presently, if a parent re-renders, and it passed a lambda to a component, that component will always have changed props. HOWEVER, this is very inefficient!
+    1) Process the file to convert lambdas to provide debug information which includes all of the values they close upon (as properties set on the lambda itself). Also an id, unique to that specific piece of code.
+    2) When evaluating if props changed, if the lambda id is the same, and the closed upone variables are ===, then... it's the same thing, so consider the prop unchanged
+    3) Test if all of our "updateOperation" lambdas in SideOpConfig.tsx work with this. I THINK updateOperation is the same, but if it isn't... we can always recognize nested lambdas.
+
+Option to not use permissions checks locks
+    1) Create a way to have any code run without locks (we might already have this?)
+    2) Add a flag in schemas to either specify all functions, or certain functions should run permissions without locks.
+        - Also something to specify a function SHOULD run permissions with locks
+    - Using the ArchiveViewer verify "heartbeat" goes from 25 to 0 locks
+    - If you step into pathValueClientWatcher.ts:setValues in FunctionRunner, you can see the actual paths that are used.
+    - Benchmark a simple function to see how much faster this is, and how much less memory we use
+        - If it doesn't matter... maybe remove the feature?
+
+More corruption resistance files
+    - Add a new serialization format, via versioning in the settings
+    - Each datum will be length prefixed WITH a special sentinel AND with a checksum for the datum data
+        - The sentinel can depend on the length (but otherwise be constant)
+    - We can have a recovery mode (automatically attempted if the checksums are wrong?) where we scan the file for length+sentinel pairs (where the sentinel matches the checksum), see which ones have a valid checksum, and then use an algorithm to decide which ones to use
+        - Maybe we sort by success rate, which is [-numberOfValidChecksumsOverlapped, length], which is very likely to work EVEN if there are values which have the right checksum, but are actuall corrupted.
+            - As long as the checksums are random enough, so... we should never use any small values for checksums, etc.
+    - Record the value layout in the settings, so we can generate arbitrary values, even if we've never seen that layout before.
+        - A series of parallel object arrays, with each object having flags (which indicate which values exist), values which always exist, etc
+            - Value types will be string, float64, byte, Buffer[]
+                - and... we might as well add support for short, int, float, and uint (uint is a good way to store a guid, via storing 8 uint variables).
+
+Schema/binary PathValues accesses
+    PER Path, not per value. Values will likely be lazily deserialized anyways, making them either pointers, or just numbers, stored in unchanging objects, which... should be quite fast.
+    0) First our stream/schema wrappers. Which at first just convert to raw path reads/writes, but once we add schemas they can efficiently create/use those
+    1) PathValue.path (string[]), need to be converted to an opaque type `Path`
+        - Helpers to manipulate it, and go back to string[], string, etc.
+    2) Have Path be able to store itself in schema + variables mode
+        - Where schema is shared across all Path instances
+        { schema: { path: (string|Variable)[] }; variables: (string | number)[]; }
+        - Add functions to access the schema and variable values
+    3) Create an efficient `Map<Path, T>`, via using schemas from the paths
+        - By requiring Path as the input we can directly use the schema to narrow down the cases, and then within that we just have to lookup values by variables
+    4) Use Path everywhere, replacing PathValue.path, using our efficient lookup to manage it.
+        - I think a lookup should handle all the cases. We should be able to nest them as well?
+    5) This SHOULD let us entirely get rid of path joining, which should be SO much faster.
+    6) Update serialization to keep schemas, instead of converting them back to paths. This should be more efficient, and a lot smaller.
+        - Values serialization won't change, and we still need to encode the variables, but... it should still be a lot faster.
+    7) Try to remove as many Path conversions to string (and also to a lesser degree string[]), as possible, by making them schema aware.
+    8) Investigate further optimizations
+        - Replacing variables with numbers, so we our internals Maps can be replaced with arrays / typed arrays
+
+    IMPORTANT! Actually... a lot of the following is wrong. We should have this PER atomic value, and perhaps ONLY for paths!
+
+    Base code
+        reader
+            let viewTime = 0;
+            for (let user of global().users) {
+                viewTime = user.viewTime;
+            }
+            return viewTime;
+        writer
+            for (let [video, viewTime] of viewedVideos) {
+                global().users[userId].viewTime += viewTime;
+            }
+
+    Schema optimization
+        writer
+            let changeStream = new SchemaPath(() => global().users[wildcard].viewTime);
+            write(() => {
+                for (let [video, viewTime] of viewedVideos) {
+                    changeStream.write(userId, viewTime);
+                }
+            })
+        reader
+            let schemaWatcher = new SchemaPath(() => global().users[wildcard].viewTime);
+            watch(() => {
+                let viewTime = 0;
+                for (let [value, userId] of schemaWatcher.getValues()) {
+                    viewTime += value;
+                }
+                return viewTime;
+            });
+        reader specific path
+            // I guess we have to support gets as well. It should still be faster, as the schema is pre-defined
+            let viewTimeWatcher = new SchemaPath(() => global().users[wildcard].viewTime);
+            let factorWatcher = new SchemaPath(() => global().users[wildcard].factor);
+            watch(() => {
+                let viewTime = 0;
+                for (let [value, userId] of schemaWatcher.getValues()) {
+                    let factor = factorWatcher.get(userId);
+                    viewTime += value * factor;
+                }
+                return viewTime;
+            });
+        delta reader (eventually)
+            // This can easily be extrapolated to just getting deltas
+            let deltaWatcher = new DeltaWatcher(() => global().users[wildcard].viewTime);
+            let viewTime = 0;
+            onDelta(() => {
+                for (let [newValue, prevValue, userId] of deltaWatcher.getChanges()) {
+                    viewTime += newValue - prevValue;
+                }
+                return viewTime;
+            });
+
+    - Can be binary, or not.
+    - Streams can work with non-streams, and vice versa.
+        - The streams need to be setup in a tree, so we can efficiently check for watchers of them
+    - We also need to support partial key watching. Often we will watch a few keys (and then within them, maybe all keys at another level).
+
+    1) START by supporting write streams (but NOT read schemas), as this allows us to define our schemas.
+        IMPORTANT! Actually... a lot of the following is wrong. We should have this PER atomic value, and perhaps ONLY for paths!
+        - This will give us a big chunk, which we will pass around (even passing around arrays of chunks).
+            - The core will break this apart somewhat, with an object per schema, and then a tree of maps for the dynamicValues inside of it (and global lookups for the locks and values)
+        - We will never create this from PathValues, instead, we will append values to specific schemas as we build it
+            - And... locks need to be kept track of as well
+        - So we need a global "captureWrites", to set the state
+            - It will return a function, which will take parameter to finish the writes?
+                - Or... something. We need to look at proxyWatcher and see what the best way to do this is. I think it might check the values before finishing them?
+        - The schema builder functions (on the object returned by defining the schema), will then internally add to a lookup in the globally capturing state (keys by the schema seqNum)
+            - Directly adding to the PathValue for that schema
+                - A lot of fields will only be set on finish
+        - I think we still need to support the helper flags (.valid). Which is actually fine, even if it's binary. It's fairly easy to flip flags in binary data...
+        - When we need to reason about PathValues independently we can provide some kind of pointer than can be used in conjuction with the chunk
+            - Although most of the time we will just provide iterators to iterate over all the PathValues?
+            - I guess eventually the pointer could be serializable too, so that we could select PathValues with a Chunk + Buffer containing pointers, so we entirely manage our own memory. But... probably not for a while, as { schemaId: number; index: number }[] should be VERY efficient to allocate and store, especially if it isn't persisted.
+        // PathValue
+        {
+            schema: ({
+                type: "constant";
+                key: string | number;
+            } | {
+                type: "dynamic";
+            })[];
+            values: {
+                dynamicValues: (string | number)[];
+                // Pointer to position in value Buffer
+                setValue: Pointer;
+                // SeqNum to ReadLockGroup (as often many values will have the same set of ReadLocks)
+                readLocks: Pointer;
+                ...more fields for various PathValue fields
+                // NOTE: ReadLocks need to reference another binary structure
+                //  - Probably via a locally unique seqNum, which is remapped upon receiving data over the network
+            }[];
+        }[]
+        // ReadLockGroups
+        {
+            seqNum: number;
+            lockSeqNums: {
+                schemaSeqNum: number;
+                seqNum: number;
+            }[];
+        }[]
+        // ReadLocks
+        {
+            seqNum: number;
+            schema: ({
+                type: "constant";
+                key: string | number;
+            } | {
+                type: "dynamic";
+            })[];
+            locks: {
+                seqNum: number;
+                dynamicValues: (string | number)[];
+                startTime, endTime, flags, ...;
+            }[];
+        }[]
+        // Values
+        {
+            seqNum: number;
+            // arbitrary binary data
+        }[]
+    2) Isolate this object structure, so it is only manipulated or read via helpers, so we can change it to use a binary format later on
+        - This means adding functions to go to/from binary, even at the start, and just having it go to PathValue and use PathValueSerializer for now.
+        - We will have unique id (a number, locally unique) per schema, so we can very quickly tell if anything might match
+        - OH! use global objects for schema defs, so equivalent schema defs share an object. This lets us store flags for them!
+            - Not the whole schema, just the def part
+            - Just a pointer as well
+            - If we have conflicts with other schemas (now or in the future), this needs to point to the shared data for the combined schema.
+        - Expose the shared objects as createdSharedAccessor<Object>(key) => { get(schema); set(schema, value) }
+    3) THEN, use this structure throughout, replacing PathValues in the entire application with lists of these chunks
+        - We'll probably store in the core in an entirely splayed manner. Maybe the schema, then a tree of dynamic values with the leaves being the set value?
+            - The set value will contain much of the PathValue, excluding the path
+        - Update the core code to store data in schemas
+            - If schemas conflict, we take the more general union of them. Annoying, but this should be rare, as schemas aren't recursive.
+                - Ex, "x.*.z" and "x.y.*" NEED to be combined to "x.*.*", at least for storage.
+                - AND, we need to store the original schemas, and have them MAP to the combined schema!
+            - So we end up with a non-schema data storage, and a bunch of schemas
+            - TWO ways to access data in schemas
+                - If you have a schema, you just directly lookup your data in the shared schema data, which is owned by the core, but every schema points to it
+                - If you don't have a schema...
+                    - We need a lookup of schemas as values with wildcard (I guess empty keys equals a wildcard? Hmm...)
+                        - I think we disallow reading empty keys? If not... maybe we should? They are already used a wildcard watches anyways...
+                    - Check each part of the path, with wildcard filling the rest "x.y.z" => ["*.*.*", "x.*.*", "x.y.*", "x.y.z"]
+                        - OH! That's not enough. Because... "x.*.z" is a valid path, which won't match any of them.
+                        - So... maybe a sorted list?
+                    - Should be reasonably fast...|
+    4) Then update it to be binary
+        - On write
+        - And on network traversal
+        - I think PathValueSerializer needs to be rewritten. Basically, it will be given schemas ahead of time, sometimes (or maybe always, as we might as well always encode in our base schema), and then encode the paths more efficiently.
+        - OH! ReadLocks need some kind of schema as well. Hmm... I guess, they can use the schema of their readers?
+    5) ALSO, create some kind of global string lookup? A lot of string keys will be the same, and it's easier to manipulate/store numbers
+        - We will need to provide definition when we send them over the wire though
+        - THIS is IMPORTANT! Without this our strings become inefficient, and the fastest code would involve mapping all strings (such as userIds), to numbers at an application level, which is a lot of work that the framework should really handle...
+        - Maybe not for ALL string keys? Hmm...
+        - OH! And... they only need to be unique per schema? Hmm...
+        
+        - Test with structures like "components.*.x" (but lots of different x values)
+            - Hmm... the big speed issue is that the component ids don't map to an array. Although, in theory, we could do that, remapping ids to indexes. And then... reuse indexes on gc, so they become direct offsets into binary memory.
+                - Then... I guess we can remember the last mapping, and so quickly go from string to index, with just a === on the strings (which should just be a pointer comparison), which... and even though the index is dynamic, that's still basically just a pointer. So... we could recover a lot of the speed
+            - Maybe we should try this out in a mini-benchmark, with a greatly simplified structure (values constant size, no extra fields, all schemas are just 3 long and have similar structure)?
+        - AH! Have the schema context wrapper (so schema reads/write get registered), give us a context object, which it gives back for the same type of access.
+            - THEN, we can have state per watcher, and... maybe per watcher+schema. Then we can use this to cache the last keys at indexes per schema, per watcher... and for component accesses... this will always be the same, making the mapping instant every time
+    6) Support reading from schemas as well
+        - If the data is already stored in the schema format, querying in this format should be very fast
+        - We will need a shorter read binary format, which has a schema and dynamic values
+    7) AUTOMATIC schema definition generation from createSchema (replacing the code), and replacing (where possible) the uses of the schema to use the schema definition
+        - Probably the hardest part, but... the most important. Without this our application code becomes too difficult to use
+            - Nested accesses become difficult, but... not impossible.
+            - Object assignment needs to become global
+                Local assignment
+                    let list = data().list;
+                    for (let datum of newData) {
+                        list[datum.key] = datum.value;
+                    }
+                Global assigment
+                    let list = data().list;
+                    for (let datum of newData) {
+                        data().list[datum.key] = datum.value;
+                    }
+                Schema based
+                    let list = data().list;
+                    for (let datum of newData) {
+                        listSchema.write(datum.key, datum.value);
+                    }
+
+                Reads are more likely to use highly nested assignment... even cross function
+                    - Cross function stuff is hard.
+                    - We might need to specialize functions?
+                        - As in, per type of possible schema input object, make a new function, and still leave a non-schema function.
+                        - Actually... this should be fairly easy. We aren't going to use === on the functions... ever? And if we do... then I guess it'll fail (or they can mark them as non-specializable, or... we can detect if === is ever used on the function).
+
+    8) Delta watchers
+        - If we store change reasons in the schema format... then we can query deltas for free
+            - Stored in the packed format.
+
+Global nested object state for latest values
+    - Can have multiple global objects, maybe one per collection / schema?
+    - We still need to register reads
+        - If we can get code to not write duplicate reads (which a lot of code can be verified to not do anyways), then tracking the reads becomes very simple with our binary schema data, basically just involving writing numbers (or even just the start/end of ranges) to an output Buffer.
+
+Code transformation for accesses
+    - Add code which can parse a function, find any schema accesses (this requires walking the type tree, and maybe checking other files), and transform the code to call data.q() functions instead (if the schema is a schema2 schema)
+        - Add support for __callerFileName__, etc so this function can determine the caller file, so it can know how to parse it.
+        - At first do this via a stand-alone function call?
+            - And then make it a parameter on the schema?
+        - OR, should we transform the code directly?
+        - If we do it at runtime, we have to figure out capturing scope again. This is hard, and breaks static variables, but... maybe that's okay?
+            - We could always warn about static variables
+            - For any non-static variables (constants, functions, etc), we will need to duplicate them, and anything they access, etc).
+                - THIS is actually REALLY nice. We can add a file name for this so it can be discoverable in devtools, or, the user can set `debugger;`, at which point, they will see a file which just has the function and anything it depends on. This makes debugging easier, as it removes everything not involved in that function (and more importantly, shows everything that IS involved, which could be surprising, as you might miss the fact that certain functions are being called, but if their definition is there you are far less likely to miss it).
+
+Code transformation to turn types into schemas?
+    - If we could do this... we could get fast behavior from normal typescript code
+    - We would need an out in case the code accesses the schema in a non-schema type way (just for accesses, the backend should already be able to handle this).
+
+Binary ValuePaths, which are never decoded
+    1) Allowing accessing a value in a way which returns the intermediate deserialized object (the Buffer[], strings lookup, etc) PLUS the id (or an object?).
+        - Call it, ValuePathBinary
+    2) Allow using ValuePathBinary as an input to SchemaPath parameters, updating the SchemaPath code to directly copy the binary data
+    3) Create a function which exclusively accesses ValuePathBinary, using it for nested sets (getting a value, using that to index another value, etc)
+        - Verify we are never actually decoding the values
+    4) Hack together something where we can NOT decode strings in the ValuePath serializer, sometimes, as our special function will never use strings directly.
+    5) Benchmark to see the benefit (probably via memory pressure), or not decoding the strings?
+
+Automatic commit wrapping of local synced writes
+    - If we write or read from local synced state... and are not in a transaction, we should start a transaction, which exists behind the end of the call, and is only stopped:
+        a) After Promise().finally()
+        b) When any non-local synced state is accessed (which will cause us to throw as well)
+        c) When any transactions are explicitly started
+    - This resolves the issue of having to constantly wrap state writes, which is just annoying
+    - We SHOULD wrap this into a singleton local watcher which we re-use for local events ("once" watchers). This will make events faster, and should make it easier to implement.
+ALSO
+    - Allow accessing local state values without being in a synced state?
+
+Server crash log protection
+    - Up to 10 times per hour, synchronously write unclassified logs to special single log blocks
+    - We will delete these special blocks when we finally write the logs
+    - This way if we get a fatal error, the console.error before we crash (which HOPEFULLY) we get, will let us know what went wrong.
+    - OR... maybe not... maybe the server manager should just watch for crashes and log the most recent stdout/stderr logs?
+
+PathValueController direct remote database support
+    --ext-remote "D:/code/other/loader.ts"
+        {
+            /** undefined means it doesn't need to be loaded remotely */
+            loadRemoteValue(path): Promise<unknown | undefined> | undefined;
+        }
+        We'll probably have a timeout, just providing epochValues if loadRemoteValue takes more than a few seconds
+    We COULD allow for core to be resolved to the same thing independent of version (by making the core register itself globally, and all cores to use the same thing).
+        - AND, we could do the same thing for a few other files, such as clientWatcher, etc
+        - This would allow clientWatcher.setValues to be called explicitly
+        - This MIGHT be useful, but... then again... it adds a lot of complexity
+
+Hot render throttle
+    - We want all inputs to be hot, BUT, to not render on each keydown. So... we need to delay rendering triggering?
+        - BUT, we run into issues if a button then runs, which expects to have the most recent data closed upon? Ugh...
+
+Auto commit mode
+    - If anything is accessed outside of synced state, enter a commit, and end that commit when:
+        a) Promise.resolve().finally
+        b) We try to end any other commit
+    - Should be a Querysub config value which is on by default
+    - VERY useful clientside
+    - We need to support "lazyClose" watcher in proxyWatcher, that DOESN'T reset the runningWatcher UNTIL it is explicitly reset, OR, until another watcher tries to run
+    - We need to detect synchronized accesses in proxyWatcher and start this special type of commit
+    - We need to know if we are in this state, and THROW if we access any non-synced paths
+        - This is mostly for non-local paths
+        - Throwing WHEN they paths are accesses is required, otherwise the error location is too annoying
+
+Watcher diff mode
+    - Running stress tests (satTest.ts) with a single watcher shows that the slowest part is our function watcher, specifically all of the parts that don't operate in a delta mode (ex, setWatches).
+    - If we made a watcher fully support a delta mode (which is fairly easy, as it converts all the changes to deltas anyways), the watcher would likely work MUCH faster.
+    - ClientWatcher needs to expose a delta interface
+    - proxyWatcher also needs a delta mode
+    - Rerun the benchmark after using the for FunctionRunner, and our call/s should go up significantly
+
+Shard display / manual sharding / automatic sharding
+    - Sharding will be fixed on process start
+    - We can augment our database path size distribution display to also support explicit sharding control
+        - This means we have all machines register, and then tell them what pathAuthorities to run (generally just running a single process, but always a new process, with us never changing pathAuthorities for a process/node)
+    - THEN, we can write code that automatically sets the sharding configuration, not changing it too often, but keeping everything fairly well distributed
+        - Our merging should automatically handle cleaning up dead nodes, so we really just need to create and kill processes to control sharding.
+
+Function implementation benchmarking / profiling / diagnostics
+    - Have a debug page which shows all functions that have been run, with diagnostics
+        - Time taken (total, average, etc)
+        - Lock count
+        - Rerun count
+            - Really important, as it is easy to right functions with a high rerun count. Ideally every function should run at most 2 times, once to know which path it requires, and again once those are all synced.
+        - Reject %
+
+Fix wildcard permissions
+    - Wildcard values should do more than just check "", they should check all possible direct permission keys
+        - This fixes the `{ admin: { PERMISSIONS(){ return users()[config.callerId].isAdmin; } } }` type of permission check
+            - Also `{ serviceSecrets: { PERMISSIONS(){ return false; } } }`
+        - It isn't SO bad, as Object.values() only provides shallow values, but... we should still fix it.
+
+Browser local PathValue caching
+    - https://rxdb.info/rx-storage-opfs.html
+    - Do it at a core level
+    - Start with being able to enable it for specific paths, triggered by a flag to createLocalSchema which causes that entire schema to be synced.
+        - Or... createLocalSyncedSchema?
+    - Eventually add support for remote paths, but in a way to not erronously trigger the sync flag
+    - We need to batch the PathValues that we store, and load them in efficiently, etc, etc
+
+Combined console/event display
+    - Everything grouped by event, with minor sorting based on the latest events, but it is mostly just an aggregator
+    - Show the most recent events
+    - Allow drilling into a specific event, searching it, filtering it, etc
+    - Allow setting up debug workflows
+        - Take a specific event / search, run some code on it to filter and create another search, which gets values, which we then can run another set of code on, etc, cascading
+    - Tree summarization / navigation
+        - For tree display, take any node that has > N children (maybe 100?), and wildcard it, collapsing all children into a single node (and merging all child tree, etc, basically rewriting it so instead of many keys, it has exactly 1 key). This should give us a small number of finite paths. Then we can drill down far enough to split each node into large enough chunks (so we aren’t only looking at roots, but also not only looking at children), with the split factor being configurable (so we CAN just look at root, or just look really high level).
+
+Heap analyzer
+    - Start by showing shallow size with a depth of 1, and then allow clicking to add more depth
+        - Maybe allow viewing the options at each level (instead of just taking the first reference)?
+
+Better serverside logging
+    - Batch it or something, so 20 requests per second (which isn't even that much), doesn't cause the console to be unusable.
+    - Maybe... group by type, and show all the categories on the screen at once, allowing the user to type into the console to expand a group, pin a group, filter, etc
+        - The most recent groups can be shown first, WITH, extra prioritization for severity
+        - Expanding a group will show the specific entries (otherwise we just show the a count, and MAYBE the latest value?)
+        - Filtering would probably be useful too
+    - OR... we could just show a really simple log, and link to a web UI?
+        - In the web UI we could show logs for multiple processes too!
+            - We could tag groups with processes that contribute to them
+            - And also show active processes, and recently terminated processes
+
+Cloudflare bootstrap caching
+    - Add round robin DNS entries (or just verify we are already using it), for our root domain, and turn on some level of caching
+        - I think we return the correct e-tag, so we should be able to cache fairly heavily
+        - If we turn the file download to be 2 stage we can cache the second stage forever (including the hashes we obtained from the first stage), which... SHOULD be a lot faster (the first call will be slower, and two calls are required, but the second call should be always cached in cloudflare, and then in the user's browser).
+    - Fix clientside node routing
+        - Update our certificate generation to have one level of wildcard.
+            - I believe (we should verify this), you can't have multiple levels of wildcards, so... this certificate WON'T be able to impersonate our full node domains (which are 3 parts), and even if it can... we do extra verification of the public key, so... it would actually be fine.
+        - Publish A records for our machines
+        - Convert the node ids to just use the machine id clientside, allowing the client to pick any Querysub node
+    - Have the node list accessed via HTTP, and cached for a few minutes, as it will change constantly, but... clients don't need the latest version
+        - I GUESS after the first read, if we can't find a Querysub node, we can add some flag to disable caching for it, which would eliminate the lag between fixing all the servers and the site being usable again
+
+Querysub node sharding
+    - Presently we allow any client to use any Querysub node. We might want to shard them too? Or at least make it allowed?
+
+Archives disk caching
+    - Cache backblaze files on disk.
+    - Only files that are at the file size limit and old enough (coordinate with the thresholds the merger uses).
+        - At a certain point the merger will stop touching files, or touch them less often, and so we will be able to cache them on our disk for a while.
+    - Limit our size to a fixed disk % (probably 5%?), with a configurable list of disks to use
+
+Clientside sourcemaps in error callstacks (maybe just have Querysub install the error stack library by default, in the browser?)
+
+typenode improvements
+    - Log typenode timings after setImmediate, top 10 + remaining timing, in a table
+        - Also, maybe entries with time > 10ms (including remaining), and not logging the table if this would make it empty.
+
+Tool to suggest balanced path ranges (based on data size, not access patterns).
+    - We can even just have it be a function, which we call on startup (telling it the count of nodes, and which index we are), and which caches the shards. It isn't live automatic sharding, but it is technically offline automatic sharding, which isn't too bad.
+
+FunctionRunner automatic balancing
+    - Exposed functions will also define define their shard distribution.
+        - Ex, setShardDistribution(addToUserBalance, (userId, balance) => userId)
+        - It is somewhat important for the shard distribution to be consistent across various functions.
+        - If this is just set on our most commonly called function, sharding should work quite well
+    - IF a shard becomes too far behind, other nodes will pick up the slack, with the closest shards picking it up first, then those farther away, etc, etc.
+        - Hopefully we can use the shard values to prevent too much duplicate work, even without any direct communication between the nodes
+
+Turn on and test backblaze storage
+    - Create a utility to go from disk => backblaze (and might as well go from backblaze => disk)
+    - Test starting a server on another machine
+    - After setting up the .json keys files, it... should just work?
+
+Anti-rejection code (isn't REQUIRED to make the database useful, so we should wait. Would be pretty slick though... Ideally this can all be extensions that have no or only modular impact on the core functions, or even no or modular impact on the proxy? Although that might not be possible...)
+    Summary
+        Excess work (N^2) due to rejections
+            - Past time reading (removes causality guarantee, almost completely preventing rejections)
+            - Unsafe reads (to remove locks, and then selectively add them back)
+        Jitter (due to rejections)
+            - Pase time reading (even further in the past than required, to allow a buffer of values)
+        Jitter (due to varying client lag)
+            - Past time reading
+        Lag makes player near impossible to kill
+            - Adjust writeTime to stay > min value, depending on the values changed (some values have to be < 100ms, etc)
+        Laggy player does things only they can see
+            - Adjust age of past dependencies to stay > min value, depending on the values changed (some values have to be < 100ms, etc)
+        Clientside operation lag resulting in inability to perform delicate operations (ex, shots miss)
+            - Past time reading
+        Clientside operations applied out of order
+            - More consistently adjust writeTimes (instead of one some functions but not others)
+        Operation cause-and-effect no ordered (phasing through objects)
+            - Stop using past time reading
+        Clientside hacking script to automatically react to values in the past
+            - Serverside changing of time, rejecting clientside predictions, AND, something to tween to the correct state, to prevent snapping
+
+    todonext
+        brainstorm: `So what is the FAIREST way to prevent lag from HELPING.`
+
+    == Test app, with top down "shooter"? (with just circles and lines?)
+        - Movement will be "key" based, not frame based. This is harder to implement, but takes a lot of pressure off of the synchronization code.
+            - And... the look position will be... just blurred, using maybe 3 ticks per second? Ugh... that will work, for now. Eventually the synchronization code SHOULD be fast enough to handle 30 ticks per second, but presently... our overhead is just too high, and so operations that SHOULD take ~10ns take ~10us, and there's not much we can do about it until we use deltas everywhere, and replace our usage of proxies.
+            - This makes the physics system harder to calculate. Although, not that much harder, as we can still use the "intersection" approach, if we want to, by locally emitting positions per frame.
+                - I'm not sure the best way to do it... intersections (which really decomposes into line segments) is nice, as it allows infinite precision.
+                - Of course, if we have any concept of "gravity", then it is harder. BUT, even then, we can still extrapolate, and even better, we can extrapolate with non-linear segments, which is really the best way to do those kind of simulations anyways.
+                    - We WOULD require a way to make the non-linear segments update consistently, but... as we have a universal time, that wouldn't be so hard...
+            - WELL, no matter what we need some kind of snapshots, to prevent having to re-simulate TOO much state.
+                - So... we can have "keys" only last for a certain period of time, and after that, movement stops. Of course, if we had gravity this would make it possible to freeze in mid-air, but, eh... it is what it is.
+                    - AND, if we had any NPCs or AI, those would require an app server, or some kind of general purpose interval function calling server anyways...
+    == Add AI to automatically move around nodes, and automatically shoot
+    == Add multiple nodes, with various latencies, which flucuate with different magititudes
+        - We should see nodes jumping around
+        - It will be hard to hit laggy nodes, as you don't know where they are!
+
+    Delayed prediction rejection
+        - Not sure if this is needed, but in theory if there is contention a function might always require multiple runs to be correct. This means our prediction will always be rejected. BUT, it is probably pretty close, so... we should just keep it around for a bit, until FunctionRunner finishes up the function, giving us the most up to date state.
+
+    Past time client reading
+        - Intentionally read client positions older that the latest, in order to cause a smoother state.
+        - Will read a combination of a fraction + value in the past (so clients that are 1000ms +/- 500ms
+            still look like they are smooth)
+        - The readLocks will have to have an empty time range (as we KNOW their range will have invalid values)
+            - At least for write values. For the UI, this doesn't matter, the UI is readonly
+        - Bakes in these values to the function call when asking the server to run the function
+            - When we evaluate the function call, if those paths are read, we have to adjust the read times
+            - OF COURSE, only if those paths are read with the request for "past time client reading"
+        - Allow CUSTOM limiting (we always limit to a few minutes), of time in the past, adjusting the read to be more recent if it is too old (by the Querysub, on received time)
+            - This is important in competitive games to prevent things from undoing a lot of other state, just because one client is lagging badly.
+                - TWO settings, one on a global level, and another on a path level, inside of FunctionRunner (using the writeTime as the base)
+            - MAYBE we actually limit read times based on the writeTime, which we can use to estimate the lag. Ex, they can only read back as far as 4X their current write lag? The default won't be to do this check, but... for competitive situations we do want to do it...
+        - The writeTime will still be the client writeTime, so by default we preserve client side-effect ordering (ex, cast "shield", then "fire", so the shield protects against the fire)
+
+    ++ Fixes nodes jumping around, as well as making it hard to hit laggy nodes
+
+    == Add a check so that dead nodes can't shoot anyone (which we might already have as check?)
+    == Laggy clients will be hard to kill, because by the time they know they are dead, they will have killed their attacker in the past, so they won't be dead anymore!
+
+    Ignored client write times?
+        - With support of "past time reads", we can always change the writeTime (somewhat, we will have to add a special case where all reads are now forced to be past time), without causing a rejection.
+        - This would need to be down on a function level (instead of on a path value)
+        - This would allowing resurrecting bad values, as well as changing the order of effects (ex, shield, then cast fire. BUT, if shield is run at server write time, it could happen after the fire, and so the fire could affect you even though you KNOW the order was correct).
+            - Which... is fine, we just have to make it clear that this reorders the function implications
+            - ALSO, make it clear that this can cause REALLY bad rejections. So if this is done with movement... the client better have really good rejection handling, or it will jitter like crazy
+            - ALSO, if the time of our write is important (ex, if we store keys, and then your position is your timeDelta * velocity), then this breaks that (so it isn't just order sensitive, but time sensitive values it impacts)
+    Expose writeTime to functions
+        - For example, so they can give some leeway in deaths:
+            `if (isDead && isDead < writeTime - 50ms) return "deadCantShoot"`
+
+    ++ Fixes making it hard to kill laggy nodes (by making them shoot at the serverTime, not the clientTime, so they can't surprise clients with "you're dead")
+
+    == Add "gravity" power, which either accelerates all users towards or away from the user
+    == ACTUALLY, I think the problem will be drift. If we are constantly writing, and assuming our prediction are correct, and having them read the previous predictions, we can get really far off from the server
+
+    Client cascading call resimulation
+        - To prevent drift, we can rerun client calls after a rejection. This is a bit annoying, and requires some way to hook directly into rejections (which... actually isn't that hard?)
+            - Basically, instead of clobbering all ReadLocks on call predictions, we identify any ReadLocks on other calls, and then watch them explicitly for rejections. If they reject, we re-simulate our call, writing in such a way that we force our previous call to be rejected (which then triggers ourself for future calls, etc, etc).
+                - We might need to write to versions MUCH farther in the past, as now we need to increment the version, so... yeah...
+        - ONLY if we don't have our result, of course, otherwise we would have no reason to run...
+        - Probably do this in a batch, which considers all calls after rejections, then deep compares values, then follows the call chain to get all candidates for rerunning (and the order they should rerun)
+            - We then need to wrote the original PathValues per result, which... is fine.
+        - ALWAYS update all the way to the latest, OR, doesn't update.
+            - Add a kind of throttling so we don't always update to the latest, to prevent lag causing huge amounts of cpu work. This will cause large "snaps" when we do resimulate, but... it will prevent the browser from locking up for too long?
+                - Set a constant factor (maybe... 60? as 60 * 16 is close to 1000?), of our extra call rate. Then it is relatively easy to throttle, basically, every resolved call value (they are all rejections) adds +60, and every rerun (each value different causes reruns, but can cause many, if there are multiple calls that depend on it) takes away 1 per call.
+            -Make sure to measure this time too, as it will be interesting to see how much time it takes
+        
+    ++ Fixes drift
+
+    == Will there be a "jump" when a user stops toggling their power? If we are only looking at their keystrokes in the past, our predictions will always be off, but it should be relatively smooth?
+
+    == Add collisions
+    == After having very few users this becomes intractable, due to the inefficiencies of calculating collisions
+
+    PHYSICS HELPER
+        - Takes many path+shape+times as input to seed the data
+        - Expose a function which takes a shape, and a range of times (as we don't know which path we are reading, so we don't know which past time to use), and gives all paths which might be close to it
+        - We then take the paths that MIGHT intersect, and do full collision checks
+            - We can do it with "past time" reads or not
+                - "past time" reads allows object to phase through each other
+                - regular reads causes more rejections, which can cause more server load, and clientside jitter
+                    - We should make it so one half of the world does regular reads, and the other does "past reads", and see the differences
+                        - Well, the difference should be, the "past time" world has no backtracking, so if you do a jump puzzle on your end, you did it. BUT, the "past time" world will also have more objects phasing through each other, and you will see other clients jumping on invisible platforms, etc, etc.
+                        - The "regular time" world will be more consistent, but sometimes you will swear you did something (jumping on a platform), but then you will backtrack and fall through it, because it WAS broken before you could jump on it.
+        *** Implement Local cache helper ***
+            - If we create a watchFunction with proxyWatcher, we can make the output synchronously accessible to other functions, as a way to allow unsafe reads
+            - If anything accesses this AND the accessed watcher has never been fully up to date (if it has been, but now isn't... allow it, I guess?), then the accessor is not fully up to date as well. It doesn't tracks it as a dependency, but instead as a "watch"
+                - AND, when the cache is up to date, the watcher will rerun
+                - AND, if the server does the same thing, we will USUALLY result in similar values, BUT, when we don't the server only runs the function once, so it will decide the final result.
+            - For things that matter, such as shooting a player, we will want actually dependencies, but this can be used to limit the things we access which we have to depend on
+            - Can lazily run, and just uses the database, so it will run in isolation fairly easily
+                - When accessing the result, sometimes... register the accessed values as being used (so they aren't unsynced). This might be annoying, and require some batching / delay, but... it is very important!
+
+    ++ Makes system a lot faster, by reducing rejections that don't change the value, and by increasing the efficiency of comparisons
+
+Automatic large value storage separation
+    - If a value is > ~10MB, when we write it to archives, we should ALWAYS break it into another file
+        - Have this decided via a flag on the PathValue, which can be dynamically set via size, or explicitly set.
+    - We should lazily read these values
+    - Garbage collection them after a while, when the file referencing them disappears
+    - We should create a cache of these on disk as well, so even if we do need to read the values, we can often just risk off of the local disk
+    - This will be annoying, but... if we already have disk value offloading, it shouldn't be so bad... AND, could really reduce startup time
+
+Archives disk cache value separation
+    - If we already support large value separation, we can also separate values from archive value disk cache
+    - STILL via some limit
+    - Because it is on our own disk, we can seek inside the files, reading JUST the part we want, so the limit can be much lower, probably 1KB?
+    - This can reduce startup time even further
+
+Network visualization
+    - NONE of this will use our PathValue system, and will instead use specialized instrumentation functions
+        - Will still be realtime though, and show paths, etc
+    - The nodes, and paths, in the browser, with information on time alive, etc
+    - Traffic information
+
+Multi domain support
+    NOTE: Special case LOCAL_DOMAIN, so that we: 1) don't trust anyone else's LOCAL_DOMAIN, and 2) assume no one else trusts our LOCAL_DOMAIN
+    - If you call a function on a cross domain, it won't execute inline, but instead result in a write
+    - rootCertDomain needs to be removed and updated with a dynamic domain
+    - NetworkTrustController needs to maintain trust per domain
+        - Only allow readLocks on paths in trusted domains (otherwise throw, not even committing the write).
+    - Writes to domains that don't trust us readLocks sanitized, so they don't depend on any of our domain values, otherwise our writes will become rejected immediately
+    - Ensure we are still ignoring writes from nodes that aren't the authority on them (I think we are, but this becomes even more important, as now we might partially trust a node, accepting some values, but not others)?
+
+Querysub bootstrapper + git repo file server + local file system server
+    - Basically... instead of requiring a static file server, we can have Querysub automatically host the files / repos based on some data written during deploy (probably specified via a function that is called in something the deploy.ts imports.)
+    - Querysub won't even know which servers it is hosting, and just host a generic bootstrapper, which when run on the client will just do a regular data access with the domain + path to get the entry point
+    - Querysub will basically just have a utility to go from gitrepo => file WITH, load development file system support.
+
+
+Automatic schema usage/generation
+    - Parse the types to generate the schema, then automatically update accesses to use the schema object
+
+PathValue.value disk cache
+    - If an entire server (PathValue server and Querysub) NEVER uses path values, we can move PathValue.values onto the disk, only  reading them off disk when we send values over the wire
+    - This will have to be done explicitly, via a setting in AuthorityPathValueStorage
+    - The values will be read off disk by our serializer, which will ask AuthorityPathValueStorage to read all the real values off disk
+    - The FunctionRunner SHOULDN'T need this, as all of the values it stores should have been used semi-recently...
+        - Actually, only really only the PathValue server requires this, as otherwise we discard values after ClientWatcher.WATCH_STICK_TIME, which is presently 10 seconds!!!??? But even at a few minutes, that value won't be too low. What are we going to do, render 2GB of images at once? (and if it is video we need to stream it anyways, likely from another storage source, as there is no reason to use PathValues for videos...)
+
+Optimizations
+    - Client storage of values, allowing the server to only send Time for large value the client doesn't have
+        - The client can just skip adding the PathValue, BUT, keep track of the last thing that is waiting to add. Then if the value is superceded anyway, it just won't add the full value, after it receives it.
+        - If the values are VERY large the server might be storing them on disk, so this allows the server to avoid even reading them off disk, which is extra efficient.
+    - One time subscriptions + polling subscriptions
+        - For large data that changes frequently this might be better?
+    - Sending very large values seems to cause something to block for a really long time. Not sure what, but... sending 100K writes with writeOnly, and dontStorePredictions, locks up for minutes at a time, then allows a lot of values to get through, then locks up?
+        - 10K writes is a lot better, and 1K writes is really fast.
+        - Maybe... writes on the websocket are buffered, and then the socket disconnects due to having too much send at once, and only reconnects later due to some random polling?
+        - MAYBE we just need better server enforced throttling?
+        - OR, if the problem are that the funtctions are getting rejected (which I am 99% is the problem), we should support "updateWriteTimeIfNeeded", and see if that fixes it?
+        - AND, we also need to handle streaming the updates better in FunctionRunner. It is kind of just batching everything, when it needs to wait (maybe with an "io" delay), to let the outputs clear?
+    - Delta supporting watcher
+        - Take all places that iterate over all paths, and have them work on deltas
+        - The callback will still have to process the changed paths directly (although it will have access to any state), and indicate the deltas to the watches
+        - This could be used in FunctionRunner to make it more efficient
+    - If we track when a path was created, wait longer to archive (2.5 times MAX_CHANGE_AGE), and then detect that is was deleted < MAX_CHANGE_AGE * 0.5 after it was created... we can avoid archiving it. But... then we need to clean up the tracking of when it was created, and that's a whole thing, so... it's just difficult...
+    - The server should batch forwardWrites when it receives them. That way if it is lagging due to a slow (synchronous) operation, a lot of values will buffer up on the network (but the batch queue will be empty, as we will have synchronously emptied it), and then when we finish the request we will have the batch time to gather up all the network requests (which should be a lot! as we can handle a lot of bytes in even just 10ms!).
+        - Of course, this adds ANOTHER 10ms delay, but... that should be fine, we won't get forwarded values often...
+    - We don't need to send back predicted values, when the prediction is accepted
+    - Client side we should only trigger watchers when a watched values actually changes (not just when there are any changes on the watches paths, at any point in history)
+    - Client syncing thrashing
+        - When we no longer need a value, we should stop immediately clearing it, and just mark it as unneeded, and then clear and unsync it a bit later on. This will take more memory, but allow navigation to be quite a bit faster
+    - Optimize parent accesses to not do brute force `.startsWith` searches, and instead maintain some kind of a lookup?
+        - Maybe... although, is this even much faster? It will certainly slow down writes.
+    - Optimize readLocks
+        - Each write stores readLocks very redundantly, making each function have `WRITES * READS` readLocks, which takes a lot to store, and requires a lot to synchronize.
+        - We can probably store a lockHash, to quickly detect duplicates
+        - Wire calls can use lockHash to depopulate duplicates, then re-populate them on the otherside.
+            - Such as commitWrites/forwardWrites, etc
+        - Our storage can depopulate duplicates when storing as well, maybe even storing readLocks in another file?
+    - Clear old pathValueProxy cache
+    - Better object comparison in certain values when we compare PathValue.value values? (only really clientside, I think?)
+        - As well as more object comparison checks, to not notify if the value is set to an identical value.
+    - Streaming / waiting during storage operations
+        - For example, we can stringify the file in chunks, instead of all at once, that way we don't block for > 100ms. We should wait ~10ms, giving a lot of requests time to clear.
+        - This isn't going to be especially big for large datasets, which may require merging everything at once, which with our current algorithm could take seconds!
+    - More efficient wire size and serialization speed
+        - If we write to 200K number values, which all depend on the previous states of 200K values, we end up sending 94MB, which takes 1200ms to serialize. We also receive a lot of data back for all the valid states. It might not be easy, but... it would be nice to optimize this, in some way.
+        - Probably via schema handling?
+        - ALSO, 94MB exceeds our default packet limit, so it doesn't even send! We could fix this with compression, but then it would take even longer to serialize!
+
+Forward retry code
+    - We should maybe retry to forward writes if we can't forward them (and we aren't an authority)?
+
+Wait a bit before loading values, in case a server wrote values, died, but the remote storage system isn't serving the read back yet?
+
+Changing authorities bug
+    - If an authority changes, it is possible for it to receive values it doesn't care about, and then respond saying the results are valid. However, they aren't, instead it just ignored the values.
+    - ALSO, it is possible we will send a client ends up watching valid states from multiple authorities, due to being unsure if any are down, etc.
+        - AND, we could end up subscribed to multiple authorities for a single path (due to watchValueLocks), which will result in thrashing.
+
+Prediction limiting / server overload protection
+    - If the PathValue server gets too far behind, we should stop predicting values, and start waiting for updates until the server can catch up
+    - This can also help when the server goes down, OR, more likely (hopefully), when the client's internet goes down.
+    - ALSO, the first thing to try is really to try switching servers, which will give us a form of automatic load balancing
+        - Although... if the limiting factor is our internet... then switching servers will actually make it worse. So... idk... this is actually difficult!
+        - MAYBE the server has to ask us to switch servers, and all we do when we find slowness is to slow down our writes?
+
+Clientside function threading
+    - Clientside it would be fantastic, as it would allow render functions to NOT block the render thread. This means we could run multiple threads, keeping the application responsive even if a very slow component is rendering.
+        - We can convert dom event callbacks (or any closed values) to { functionContents, closedValues: { [variableName: string]: unknown } }. If the variable values are either serializable, OR, can be linked to exports (ex, misc_1.isEmpty), then we can recreate an identical function just by using eval.
+            - If we do this, we would we ALWAYS want to recreate the function. This ensures there is consistency if we call the callback cross thread, or on the same thread.
+            - We can store this style inside of the synced state, and make it available to be called directly.
+
+Serverside function threading
+    - Required to kill very slow running functions
+        - They will of course we able to change their default timeout time
+        - They will need to be given an error result, to prevent them from rerunning
+
+Serverside rendering
+    - Create a mounter which can go from vnode => string
+    - Clientside, probably render from scratch, and just have our clientside mounter check for existing DOM nodes (probably just when instructed to, which will happen on initial render, maybe... always on initial render?)
+        - Events COULD be an issue, or... we could just assume any unknown nodes have no event handlers?
+        - I THINK we can probably ignore events before we render, at least for version 1. The app is mostly a SPA anyways, so the time before render should be VERY short, and then everything should involve re-renders and NOT re-navigations. For version 2 we can capture everything globally via a built in script tag, and then re-trigger them once we actually mount (and either block, or delay links, so we don't get in a state of constantly trying to load/render, getting 50% of the way there, but then the user navigates via a link).
+
+TESTS
+    - Tests which track the code used so we know which tests need to be rerun
+    - Render tests, with render JSX and paint it
+        - Can use a partial JSX => paint cache, so most of the time if the output doesn't change, OR, if the output doesn't change enough to change how it is painted, we don't have to repaint it.
+        - Also, a test harness that can click on vnodes and simulate their changes would be incredibly useful!
+            - This would get rid of the chrome javascript runtime step. We might still use puppeteer, but also... it is possible we could just use blink, or some other library, to just go from HTML => image extremely fast (realistically most pages should take < 50ms to render)
+    - True test coverage checking
+        - NOOP expressions (x + y becomes 0, etc, a function call that returns a number becomes 0, etc, etc), and see if any test changes. If not... then that line does not have test coverage
+
+=== Development tools feature complete | TASKS = 22 ===
+
+Latency / load weighted node selection
+    - When they are multiple candidates, prefer the closer nodes
+    - ALSO, prefer nodes that have less load, allowing a form of automatic network balancing
+    - The ability to switch nodes would be nice as well
+        - Switching PathValueController nodes should be easy, we just subscribe everything to a new node, then disconnect from the previous node.
+        - We switch Querysub easily (using multiple), and disconnecting from the previous ones as soon as all calls on them have finished running (really that is true for most nodes, we don't want to disconnect when we have pending calls!)
+
+Speculative function calls (calling functions that haven't been requested yet, just to sync the state they might need)
+    - Will be a bit involved, and might even require some clientside cooperation (ex, to detect which actions are possibly via which event handlers are being used), but... could result in a HUGE speed improvement
+    - Will have to determine what functions are most likely to be called
+        - We will likely need to give some context information for functions, or at least know their index, and maybe even page position? And maybe even the mouse position?
+        - If we run functions calls when the user moves their mouse towards a button we can probably increase our estimate quality with little effort
+            - We should test to see how long the mouse dwells over a button before clicking. Even 10ms would be useful. And also the average mouse speed. If we find users usually click after dwelling for say... 10ms, but take 20ms to click, we can get a 10ms headstart on the call!
+    - Will re-render after the click and then pre-sync the new reads needed
+        - If we do this on another thread, and somehow set the new syncs with low priority... this should be basically free (as if the user is moving their mouse around it isn't a background task, and we can at least monopolize a single core!)
+    - Will also pre-populate the clientside function cache, so when they really click it we can get the prediction very fast
+
+Delta based render cache
+    - The best idea is to mark reads as being delta allowed, and THEN, marking output values as being reduceable. Then we just run with partial values given to the read, and merge the new writes with the old writes. IF the writes differ in any places that are not reducable... then we have to run with the entire set of data (and it results in a warning). But if not... it just works!
+        - The reduce functions can be 
+    We already know the delta when we re-run watchers (as we reran them for a reason!), 
+    - Maybe at the component level? Although... I kind of want to implement it for just arbitrary data
+    - A function will have the ability to read from an object/list with a "delta" read
+        - It will still receive the object/list, it might just partial
+        - EXCEPT, we might need to return special state for deletions?
+    - If the only reads of the object/list are delta, then the next time there is a change, only the new/changed values are provided
+    - The output is analyzed, and the differences between the new output are found
+    - For every different write, we determine the update technique
+        - This will probably involve setting some special value on the write?
+        - Examples would be "sum" or "join" or "insertSorted"
+        - The most generic case receives the previous state (that has changed, already drilled down), and the new state, and updates the previous state (in memory) so it now has the new state as well.
+        - We need to handle writes that propagate deletions
+
+Blocking FunctionRunner call mode
+    - Will probably be slower in most cases though, as each value accesses required a round trip with an authority to read the value
+        - Might be useful for code with expensive side-effects?
+    - Add a BLOCKING mode to PathValueProxyWatcher, which uses Atomics.wait to block until a read value is synced.
+    - This potentially allows for efficient function evaluation, by not requiring any code to rerun.
+    - This is only possible if something can mark a value as synced on another thread, and absolutely not if we are single threaded (Atomics.wait wouldn't even work on the main thread of a single thread app anyways)
+
+Function results!
+    - Specially marked "specializedHardwareCalls" functions will be able to "return" a result
+    - Only possible with blocking FunctionRunner mode, as for the functions we want to return results we also only want to run them once (for expensive operations)
+        - Of course, this doesn't prevent: 1) slowness cascasding syncing, or 2) result invalidation
+    - Hmm... we also want the ability to cancel calls?
+        - Maybe calls can do this without themselves?
+    - Hmm... if the calls are so slow... maybe the user should just manage a queue themselves, so they can see what is going on? It really isn't that hard, and if they have to manage special hardware anyway... then it won't like this would prevent them from just quickly scaling a function?
+    - At this point we should have support for unsafe function writes. We MIGHT want to make the writes from this unsafe by default?
+    - By default probably allow write time adjustment, to prevent invalidation (at least when we start the function call)
+    - The return object will be { type: Symbol("hardwareCall"), call: number, path: ".,querysub.com,.test" }
+        - ALSO, unioned with a type for an error { type: "error", errorMessage: string; errorMessageFull: string; }?
+    - The function return types will be modified so any callers have to handle this symbol type
+    - The proxy watcher will temporarily register to receive all calls, so it can call them
+    - The proxy watcher will search the output writes for these values
+        - Only shallowly, which is fine for now, as if they nest it I believe the Symbol will cause a throw, so the wrong value won't be written and not noticed
+    - The proxy watcher will then actually construct the call objects, telling them where to place their results
+    - When the FunctionRunner evaluates the calls, it will place the results in the specified paths
+
+Support VERY large data sets (> local disk size)
+    - If an authority hasn't been used for a while, dump the memory for it, and... become "unready". Then, if someone accesses it, go about loading it again.
+    - This only works if we are sharded, but there isn't really a better way.
+    - We should test this with some very large values.
+
+Automatic sharding of ValuePath nodes
+    - For KVP nodes as well as Function nodes
+        - More important for function nodes, but... it could help for KVP nodes as well?
+    - IF we have high contention, and many servers, the resolving process takes SERVER_COUNT * SERVER_LATENCY, which could be high. Automatic balancing can help fix this, by finding a way to identify overlapping writes via reads (ex, spatially hash read paths, and if those hashes are ===, see if the spatial hash of the write paths are as well).
+        - This reduces the SERVER_COUNT involved in contention, which... fixes the issue!
+    - We will need to support remotePathAuthority to get this to work
+    - Needs to be somewhat generic, so we can use it for both PathValueController, and FunctionRunnerController
+    - Nodes will inform their parent path nodes of their utilization, so parent nodes can redistribute child nodes accordingly
+    - Oversaturated nodes will split, pushing newer nodes down to more specific traffic
+        - Ex, from ["x"] to ["x", "x.[0 <= hash < 0.5]"], ["x.[0.5 <= hash < 1]"]. AND, when a node has enough descendants, it will stop with all child paths, and the parent will just be ["x"]
+        - BUT, we should only shard when needed. Otherwise syncing parents can become overly expensive? (Ex, if we have 100 servers, we really wouldn't want to split a small collection across 100 servers, as then it would take a huge amount of work to sync the keys, and if there are < 100 keys, it would be needlessly slow)
+    - Nodes who have parents that have insufficient members will remerge with their parent (perhaps completely taking over the parent, if the parent completely disappears)
+    - Nodes with child nodes with insufficient members will redistribute, or remerge into one path, as needed
+    - Test on a local machine, spawning many processes, killing them, etc
+    - We need to fix our path limitation thing
+        - Detect the size limit of our underlying file system, and use that
+        - This allows us to turn on long paths on windows
+            - WARN if long paths are not enabled
+        - Use this both to constrain balancing AND to have archives write to the most specific path possible, within the current limits
+
+Programmatic digital ocean droplet launching
+    - We will want some kind of administration page that the app serves by default, requiring NetworkTrust and localhost to access.
+    - From this page we should be able to:
+        - import digital ocean api keys
+        - see all currently configured droplets
+        - launch new droplets
+    - All launched droplets will be automatically configured with the code, given trust, and configured to auto restart all the necessary process, etc (so they should be able to run unattended after starting)
+        - This means we can get rid of the emails to add trust, as nodes will no longer just "appear", but instead by instructed to appear, at which point we can explicitly ask for their public key and trust it (we should never provide the key, as this is incompatible with a transition to hardware keys).
+    - We also want the ability to monitor traffic, and make recommendations about having less/more droplets
+        - These recommendation should display in the UI
+        - There should be daily or weekly emails about recommendations to some admin email account.
+        - EVENTUALLY we might have these recommendations automatically followed, for scaling, although maybe only to certain degrees, as automatic scaling can be dangerous.
+    - AND, it would be nice if we could view/administer MULTIPLE domains at once, OR AT LEAST summarize traffic recommendations for multiple domains at once.
+
+Automatic digital ocean droplet launching
+
+History for the purposes of reverting data
+    - EVEN just backups works, but... storing a transaction log is preferred
+    - If we store writes, we can re-simulate to a previous point in time. It might be slow, but it is better than losing all your data due to a database drop.
+    - If we store clobbered data as well, then we will be able to restore even if our history size exceeds the max and needs to be truncated
+    - Kind of required if we want to use it to store any important data...
+
+Fix hardcoded email and TOS accept in certAuthority.ts
+
+AST parsing to detect global side-effects in functions
+    - Global side-effects are going to break things. We should at least mark root functions that do this, OR, that have any child calls that do this.
+        - Who knows what libraries we call are going to be doing, they could have all kinds of unintended side effects involving the disk and who knows what else!
+    - The strictest mode would be the disallow global side-effects unless it is enabled per function call
+    - ALSO, AST parsing to detect module side-effects (that impact values outside of that module)
+
+Write proxy replacing
+    - If we have a lot of local writes it can get slow, due to the proxy. If we replace the writes with function calls it could allow a lot more local writes.
+
+Schemaed proxy replacing accesses
+    - Instead of passing an array, find common accesses and replace with a call to a factory function to create a schema, which generates a function, which we then pass the remaining variables to
+        - Try to put all of these beside the databaseTyped (or at least at a module level!), so we can view the code without source maps and see all the examples of accesses. Their will be 1 for each line that reads from the database (that we can find)
+    - If we set this schema in the write itself we can greatly reduce write overhead (especially if we have keys that are numbers).
+    - We can send a binary format to forwardWrites and onValues
+    - Maybe just for the wire communication. But sending a buffer as the payload, instead of a value.
+
+Fix multi authority writes
+    - Multi authority writes are annoying, because it causes issues with predictions. As in, you write, predict your value, but then have to wait because one authority is taking a long time to resolve. THEN, another write uses our prediction, but this time only uses the fast authority. This results in the fast authority rejecting our write, until we can send it the full writes, which may take a while if the slow authority isn't resolving!
+        - AND, if the slow authority takes a REALLY long time, we might never commit the write, resulting in a valid state, BUT, we can no longer commit any writes! Although, maybe this state is okay, as if we can't write the full write... we probably shouldn't write?
+        - And... maybe this whole issue is fine, as the authorities will eventually resolve, and then it will be fine...
+
+=== Shippable features complete | TASKS = 12 ===
+
+Triggers / data pipeline / autorun
+    - Basically an abstraction of what FunctionRunner does (where it watches the database for values, and then writes other values to the database)
+    - Setup via a value in the database
+        - Like a function write, but, this function reruns every time anything it read changes
+    - Will keep the same thread (maybe a dedicate thread?) for a long period of time
+    - The ability to watch data, calling a non-committed function when it changes
+    - The function that is triggered can call synced functions, committing values
+    - Will be able to sync a lot of state, and also keep values in untracked state (if it wants to)
+    ALTERNATIVE
+        - Just use FunctionRunner, and if you need to use dedicated hardware, use another domain
+
+1.3) Buffer support in SocketFunction? Hmm... I mean, if we have GB sized files, that would required... And sending a 20GB video file is maybe feasible? Hmm... or... maybe not? Maybe it is more useful for 1MB chunks, which we don't want to have to decode each time? So this is really just "increase data throughput"?
+    - Essentially, we want a JSON.stringify that converts to a Buffer, and also supports Buffers.
+    - We will probably eventually support threads, so... maybe we just use Accessors2? Hmm... Or at least some rework of Accessors2?
+    - Would also need to support it in function argument serialization / deserialization
+
+Error notification system
+    - Some kind of immediate notification + email digest system
+
+Runtime FunctionController type checking
+    - We should check the type of all arguments against their typescript type by default. That way accidental prototype contamination is less likely. And maybe there are other security vulnerabilities?
+
+Backups
+    - Taking our entire database, and backing it up periodically
+    - We can probably check the time in the last backup, and use that to infer what values will definitely be in it, using this to backup less data (nothing that is already backed up), making this efficient (to write, not necessarily to read)
+        - Once in a while we would want to write the full state, otherwise after a few thousand backups restoring could take forever.
+        - We can also be somewhat efficient about reading from the database files, using block create time to know if they are only old and already backed up data.
+    - At this point we will already have a management interface, so... this should be configurable via that
+        - We will have to think about how to rate limit it. A lot of databases would probably work by just streaming every single write, then compressing them after they exceed a certain total size.
+            - AND, when we compress, we probably want to default to the "compress old data more", so we have logarithmic seeking capabilities.
+    - Might be made redundant by the history navigation feature?
+
+=== Feature complete | TASKS = 3 ===
+
+Path queries for synchronization
+    - We could support a way to query paths (likely via a parent selector), to do synchronizations. It makes it slower for the server to continue to send values (each new value required evaluating the selector), but... it could make selecting large amounts of data (millions of values) much faster?
+
+Local network call forwarding / neighbor prediction
+    - If there is a write on the local network we can get faster then waiting for it go to the server or back (by local network it mostly means on the same machine, but different process / browser tab / thread).
+    - We will need to ingest these values as regular predictions, except because we aren't writing them we also need to watch for their valid states
+        - We also aren't getting if the write is even commited or not (which tells us if it was EVER valid), so... we basically need to assume the value is invalid if (AFTER the valid watch is done), we don't get a valid notification after a short period of time
+    - We also need to validate writes coming from neighbors, to ensure they are recent enough, and maybe we should do some other checks?
+    - Having this work for function writes would be nice, but... we REALLY have to trust the neighbor for this to work?
+        - Well... we could always run the permissions checks? In which case... I guess it would be safe?
+        - Oh, well.. maybe permissions checks aren't that important. If they are on the same network they are sitting in the same room, so they are somewhat trusted!!!
+            (And worst case they just cause the data to temporarily change, which isn't the end of the world...)
+        - AND, we can have neighbor prediction off by default, and only turned on for certain application (maybe on by default in the browser).
+
+Function call syncing (as opposed to PathValue syncing)
+    - If we have a physics based system where we predict the physics loop every tick, we can predict a HUGE amount of values (millions), possibly even using the GPU. HOWEVER, the server won't understand our prediction efficiently, and so it will need to send as all the values anyways!
+    - If instead of syncing the paths we use, we sync the functions that impact these paths, we can have a massive state be kept up to date without saturating the network
+    - We will have to be told the hash results of these functions, to ensure we are calculating them correctly.
+        - So, a server instance will have to basically watch all the paths itself, to figure out which functions apply.
+            - And... presumably the results need to be committed anyways, otherwise this is just a different database! Although... if the function runner and KVP owner are on the same machine, the transfer of paths should be somewhat efficient? And only snapshots need to be saved, so... this could work?
+            - Basically, you would run the KVP node, and the function runner, on the same machine.
+            - Some kind of pipeline would trigger the physics function to tick every frame
+            - This pipeline would be also predicted on the clientside, so it can consistent know about ticks
+            - Eventually the client will know what functions were run (it should be pretty accurate at knowing this!), And the hash results. The hash results can come later, as the client will know that it will have to simulate old physics results a few times as course changes trickle in.
+
+The ability to mark writes as "no persistent storage"
+    - Useful for things such as streaming video, which can then have a pipeline that combined it and stores it in B2 (the pipeline would mark what is stored in B2 and the urls for that content, and the rest of the data would be available on the network directly)
+        - This makes live streaming possible, but also accessing terabytes of historical data.
+        - This is different than just video uploading, which should go directly to B2, because it trickles, requiring something to aggregate it, while also requiring instant access.
+
+Read function triggers
+    - Allowing setting up a value that calls a function every time it is read, to produce the data (and non client predictable function)
+    - Allows for exposing an off-network source (like a database) to the network
+    - By default will just mark everything as valid, BUT, we could add valid ranges
+        - For databases like mongodb we could roughly do it via changestreams, to warn if the value has changed
+        - For blockchains we can perfectly detect if the value has changed
+    - For the writing to external sources side... we would need "unsafe" functions, that don't add any readLocks (so they can never become rejected). Because we can't reject values in external sources!
+
+FunctionController path syncing
+    - For small number of reads, the latency of the read is acceptable for the trade off of reduced complexity of FunctionController.
+    - HOWEVER, for functions with a HUGE amount of reads (GBs of data), we will want to sync it instead...
+        - We still need to run historical functions, so... we basically need to:
+            1) Ask for ALL future writes on a path
+            2) Ask for all writes on that path (up to what the authority stores, which is up to and including the last golden value, which should be enough history)
+            3) Sync the valid state of all writes we receive, forever
+        - The hard part, is trying to keep the paths synced simple. If we just sync every access... then we could be syncing tens of millions of paths... which means not only does an authority have to store that much data, it also has to store watchers for all of it... which... will probably be slow?
+            - Although... maybe not... it is just a Map<string, string>?
+
+Support arrays, via adding another transparent property
+    - Basically the object properties, BUT, it will specify it is an array
+    - The proxy will have to be told of this, and... function properly?
+        - I think if it's object is [] it will work? Maybe? We should double check how proxys handle arrays...
+        - Does push just map to a set plus a .length update?
+        - And what about splices!
+    - This will be annoying, and probably slow...
+
+Actually Object.keys() ACID handling
+    - Ugh... very annoying, and difficult
+    - A path authority can sync child keys for a range of time (similar to how it syncs a path for a range of time), and then compute a hash of them (sort and sha256). It won't be efficient, but... it should work.
+    - Will be OPT-IN, because the slowness of hashing the keys is almost never worth the ACID nature of it
+
+Object.keyRanges()
+    - If we use search patterns that line up with our range sharding, we can efficiently access ranges of keys
+    - We will probably need to add more details in our range sharding
+    - We might want to specify sharding characteristics via some callback near proxyWatcher.databaseTyped
+        - Ex, schema.dataSharding(database => [{ path: database.logs, map: key => +key }]
+        - Ex, schema.dataSharding(database => [{ path: database.maps, map: key => key.split("_")[0] }]
+
+Old data cleanup ability
+    - Some way to actually delete old data
+        - Maybe via both setting all values to undefined, and having nested undefined no longer stored
+
+Ability to selectively undo history
+    - Probably recursively by path
+        - We would than have to store history under those paths
+    - Probably with a limited history (ex, 30 days?)
+    - The ability to select changes by values / by time, taking everything in the same changeset (via looking at readLocks), and adding a new change that reverts the change (if the revert value is still the latest)
+    - Ex, undo all changes by user for the past day, one specified paths
+
+Disk support
+    Local disk support instead of backblaze
+        - AKA, implementing backblaze using the local disk of many nodes
+    Recent changes disk backup
+        - Store changes that are too young to be archived on disk
+            - CAN'T use archive the values, as those values are assumed to be valid, so... needs a new format, which is just used to store changes, in case of a crash.
+        - If we have a single node, we can stream our changes to the disk, reducing our data loss on crash from ~10 minutes, to < 10 seconds, with very little disk IO
+            - We can have a flush time option, which can be set even lower, which will increase IO, but can reduce losses from crash to any point we want.
+
+C++ (.cpp) import support
+    - Via WASM compilation, which can be done via https://github.com/sliftist/cpp-portable-loader and `clang-wasm`
+
+C# import support / other language child processes
+    - Would only work on the serverside
+    - Would have to run a child process which runs the C# code?
+    - If we are going to run another process, we can just run any easy scriptable language. Python would be easy to get working as well. On a smaller scale, Java would work too.
+        - Ideally we automatically generate typings, but... even if we didn't, that would be fine.
+    - C++ would be useful too! As a lot of the time this is useful for making OS calls, which are easiest in C++.
+
+Rust import support
+    - Compiling Rust to wasm is even easier than C++, so... we could probably do it fairly easily
+        - I'm not sure how to get typings, although they aren't strictly needed
+
+=== Advanced optimizations | TASKS = 2 ===
+
+=== \/ Speculative tasks \/ ===
+
+- function call results
+    - Some way for function calls to return T|undefined, and be given the location they are assigned to, so they can automatically write the result there
+        - Is basically the same as adding the path, so... this is probably a good way to do it.
+        - Can still with with function predict!
+        - Passing the path to the function call is a bit annoying, but... if we return a special value, we can observe where that gets written to in the proxy, so... it isn't so bad...
+            - We can even allow writing to multiple output locations
+        - Really only makes sense if the function is slow. Otherwise, is should just be an inline function!
+            - We should add in function level sharding, so we can exclude function calls from function runners that don't opt into those. That we can split up slow functions from regular functions.
+
+- "limits", to limit resource strain any node can put on PathValueServer
+    - limit the total number of paths watched
+    - limit the number of writes per second
+    - limit the total bytes per second of writes
+    - maybe limit onValue callbacks, depending on size, etc
+    - Maybe dynamic limits, depending on load
+
+
+- Fix our JSON serialization code converting undefined to null inside of arrays
+    - This impacts function calls, which can be really annoying / confusing...
+
+- Actual query subscriptions
+    - Tell the querysub server that function we want to run, and everything it watches will be returned
+        - Anything the server misses can be additionally subscribed to
+    - This allows the cascading latency penalty to be equal to the querysub server latency. WHICH, if it is on the same server as the PathValue server, can be less than 1ms (we will need to turn off some automatic batching to lower it this far)
+        - AND, if the querysub server did a full PathValue sync, asking for all values on all paths... the latency would be 0.
+            - We would need to be able to fit the entire database into memory, but... if we allowed `watchRecursive(path)`, and either shared Querysub, OR, had queries explicitly map to the path they wanted (which hopefully would either be small, or shared with another server), then... it would just be efficient...
+    - AND, the client could even subscribe to `watchRecursive` as well?
+
+Better FunctionRunner startup
+    - We should sync all of the repos we will need BEFORE taking any shard space? As the clone could easily take minutes, which would cause functions to timeout.
+        - Although... maybe just having it better distributed would solve that problem, as other nodes should automatically pick up the slack?
+
+CORS support, so querysub can be accessed by other domains
+    - We would read the CORS values from the database, and they would be set in, or similar to "yarn shard-deploy" / "yarn deploy"
+    - Only querysub needs CORS, as the browser should know to only access it
+
+Serverside "local" writes
+    - Update code to special case "local" writes, so that we never sanitize remote writes so they never use ReadLocks on "local" writes.
+        - BUT, allow "local" writes to have ReadLocks on remote writes (IF we are trusted on that domain). Our system should automatically subscribe to valid states, and maintain the valid state of "local" writes. This prevents our "local" values from having invalid values that stick around (as in, in a cache), even though we might end up writing them to a "remote", which would make them invalid.
+
+Have deploy log the % files and % lines that are NOT allowclient
+    - This determines which files we might serve clientside
+    - We only really care about files NOT in node_modules, but, in our folder?
+        - Although, maybe we should consider any private npm packages as well?
+
+Lazy Function / Function Prioritization
+    - If we have functions that can run at any time (but must run in order), then... we can run them later, when the server is more free.
+        - AND, we can even go as far as to detect when the predicted outputs of them are needed, running the functions at those times?
+    - Maybe we would actually want this to be functions triggered one read, but only triggered once?
+
+Ability to disable syncing of paths
+    - Possibly depending on context, such as page
+    - Also, possibly just to reduce the update rate for certain paths
+    - This fixes the issue of lag (either network, client, or server), when many pages use a commonly changed value
+        - Ideally we WOULD like it to be updated, but... if we don't need it updated, and 99% of our server capacity is spent updating a few values we don't care about...
+    - We want this to be able to be added post-hoc, potentially live (at a data level, not a code level), 
+    so the site can be quickly optimized on the fly
+        - But of course, the code will also need the ability to specify it doesn't want updates
+    - Will basically function as a watchLatestOnce, where it gets the latest value, and then no more.
+
+Better undefined support
+    - We COULD make it so for the last run of a function, after all values are synced, we run again returning undefined for every value that is fully synchronized with no child values synchronized.
+        - If the first run has every value synchronized we would have to run again.
+    Pros
+        - This MIGHT make function run more naturally
+    Cons    
+        - Functions are more complex to run (and must run more times)
+    Alternatives
+        a) Just use atomicObjectRead if you want to read undefined
+        b) Use `+x`, `x + ""`, OR `${x}`, to coerce it to a primitive
+
+Support instances in our data?
+    - Benefits
+        - Instead of passing around string ids everywhere, and having programmers have to know which functions use it, you can pass around a class and just get autocomplete
+    - Costs
+        - Slow
+        - Harder to debug
+    - Alternatives
+        - Compile time instances => static function + id call
+            - Basically, we can "store" and "pass" around an instance, and even "new" it. But
+                new => create random id, and store data in static location under that id
+                function calls => call static function with instance (which is a string) as a parameter
+                instance accesses (including this.) => lookup data from root with static lookup
+            - This is harder to do, but... faster, and easy to debug.
+        - Function modules with local .databaseTyped call
+            - You can go to defintion and see all the exported functions, which are probably how you mutate the data (especially if the .databaseTyped call isn't exported!)
+    - BASICALLY, everything starts as a static call. But... those calls may wish to create class instances, and store those (ex, binding data and functions)
+    - SO, on seeing a set that is a class (ex, has functions), instead of just stripping the functions, we...
+        - Figure out if it is a deployed function (has exposedRootClass decorator).
+            - If not, we don't store it
+        - Store a new InstanceId in our schema with the data provided
+            [DomainName][ClassName].Instances[InstanceId].Data = Data
+        - Replace the write with this new instanceId
+    - On accessing a value that... looks like an instanceId? We...
+        - On reading values we use the Instances.Data path instead
+        - On calling functions we load the class from the Source info, and call it
+            - Specifically, providing a this context equal to the data value.
+
+Clientside "local" FunctionRunner
+    - In theory, some operations will heavily depend on some server function call completely successfully. BUT, also store local state?
+    - IF we never need this, then we never need to implement it. But... if we find a case where this is useful, in which just moving the data to the remote is not viable (and there is no alternative)... then... we could do it?
+    1) Allow "local" functions to run using FunctionRunner, so they can automatically rerun when rejected
+        - We will require the ability to mount FunctionRunners under the "local" domain explicitly
+    2) Allow (when specified) "local" functions to depend on "remote" writes
+        - This requires allowing syncing valid state, etc
+        - Without runing FunctionRunner rejections are usually not preferred. But with running it, rejections can result in a rerun, and so are not as bad.
+
+Intercept DB, to allow writes without actual ownership of a domain, cascading to store all values that depend on these changes as well.
+    - Maybe only useful for hacking together application, or development, but... seems like it would be pretty cool?
+
+Function call results (for cross domain calls)
+    - Probably not a good idea. Maybe... just never do this...
+    - Non-cross domain calls are all inline anyways, so this is only for cross domain calls
+    Something like: QuerySub.afterFunction(() => OtherController.queueAndRun(), result => TestController.finished(result)), which tells FunctionRunner to do some extra stuff after a cross domain call finishes.
+    - We COULD ALSO use async/await. We KNOW when the synchronous part of a function finishes
+        - Ex: `let result = await QuerySub.afterFunction(() => OtherController.queueAndRun())`
+        - As we are broken in on the machine, we don't need ALWAYS resume (although if the node is killed, and the function needs to be rerun, we would need to re-evaluate the previous call to catch up).
+        - We would need know when the await returns, wrapping it with some code?
+            - This is... impossible? Although... if the thread is CLEAN, we could just assume the next function will run, resolve, then wait for a promise, then assume the function is finished? It gets REALLY difficult
+    - This will break up the ACID nature of the function, which is fairly unfortunate
+
+Size optimization
+    - Move register calls into a .client.ts file, so browsers only need to import the interface, and not the implementation too!
+    - Better HTTPS endpoint which prebundles some calls, so we can bootstrap faster
+
+Streamlined externals, via a custom site that takes payment and handles setting up all the other services
+
+Lazy connections
+    - NetworkState creates a lot of connections, which have very infrequent traffic. Instead of having N network connections, we should allow some connections to be marked as less important, and closed when not used for a bit. Although... if we say had, 10K open TCP connections, is this a problem? Hmm...
+
+Synchronize time ourself
+    - We can't trust machines to synchronize their time. Synchronize it ourself in NetworkState (OR, maybe just by using NetworkState), and use that instead of Date.now() everywhere!
+    - We'll want to use a waitForTimeToSync() function in places to gate our major code locations to ensure we always have an synchronize time, so we can expose a getTime() function that is synchronous.
+
+Replace ALMOST all external services
+    - We can run our own DNS server
+    - We can replace firestore with our regular storage controller
+
+Very short expiry times on thread certificates, finding some way to automatically update them while running
+- At first only node <=> node
+- BUT THEN, our proxy should be able to handle updating the peer cert, as it can probably call renegotiate?
+- Our system for the client to update it's cert after the fact can likely be reused to trigger an update of credentials?
+    - I assume outstanding connections won't be killed if their certs expire, so... we would actually want to
+        add an additional check to close a connection if it's cert isn't updated? Or... we could just call renegotiate, which
+        I think serverside TLS connections can do?
+
+Public release?
+Get benchmarks on https://benchant.com/ranking/database-ranking
     - It seems like the throughput can be easily beat. We trade massive data-loss and memory efficiency for speed, so after our path optimizations we should be able to easily beat all others servers (again, because we lose data easily, and store everything in memory...)