[WIP]: Runtime to runtime communication

[afinch7]

My next logical step towards #38. This is very unfinished currently, but it currently at a state where it compiles and runs. I figure it might actually be useful to take a look at this point.

Most of the functionality isn't complete yet, but a lot of the ground work exists.

[afinch7]

Oh dear It seems that I forgot to merge fly.rs back into my fork after the last commit.

[afinch7]

I did a manual merge back into my branch not exactly ideal, but it compiles and runs again. Despite being a slight setback I'm really glad for some of the refactors. Some of those files where getting a little long. distributed-fly is likely still completely busted, but I'll make sure to resolve those issues once my changes are more complete.

@michaeldwan and @jeromegn I think that now is a good time to open a dialog about the performance concerns with these changes. My main concern here is that I'm using Mutexs to control access to StandardRuntimeManager/Runtime. Since access to both of these is required for dispatching request events, I expect this to have a pretty major impact on performance where it counts, request throughput and latency. Here are my ideas for improving performance:

• Make any changes needed to replace Mutexs with RwLocks even if it involves creating multiple separate scopes within some functions. This need for splitting up functions like this indicates that existing functions may need to refactored into several simpler functions.

• If RwLock refactors are not enough. We will likely need a system for extending the lifespan of a read lock beyond issuing a single event. We would need a way to cancel/dispose the locks in the event a write lock was needed.

The important thing here is reducing event dispatching latency(and throughput as result) even if it increases the time it takes to complete other tasks. I would really love to eliminate the need for a lock at all, but I don't really see a good way to do this. I'm still pretty new to managing memory and multi-threaded code in general, so there may be a simple solution I'm missing here. A benchmark for these metrics would also be very helpful.

[afinch7]

I went back and switch out those Mutexs for RwLocks, and setup a testing example. I see someone has added RuntimePermissions. I assume it would be fine to add in a permission to enable/disable use of the serviceRequest function I added for making runtime to runtime requests from js land.

[jeromegn]

This PR is a bit hard to review. I assume it's not ready for review since it's still marked as WIP. It needs a bit of a cleanup, our commits seem intertwined with yours and there is still a lot of conflicts.

Could you include an example of how it works in TypeScript? There might be one in the code already, but I have a hard time finding it through the many changes.

I assume this is a bit like the Web Worker API. I'm interested in how they're spun up and how they send each other messages.

[afinch7]

I don't think that it's ready to merge yet for sure, but I think that some of the refactored systems like RuntimeSelector -> RuntimeManager are at a point where feedback would be helpful.

Could you include an example of how it works in TypeScript? There might be one in the code already, but I have a hard time finding it through the many changes.

I already have a example I've been using to test my changes. The example, test-sevice-runtime.js, has been hard coded into the dns command for the time being. I plan to replace this hard coded setup in favor of a user/implementation configurable setup. I want user configuration to look something like a docker-compose file for development, so multiple runtimes can be configured to run at once.

I assume this is a bit like the Web Worker API. I'm interested in how they're spun up and how they send each other messages.

The current system doesn't really look much like the web worker one. The current system is designed to be setup and controlled by implementation code. It's designed for running somewhat trusted js/ts/wasm code as a internal service. With things like compilers, language services, production/staging debugging tools, or other js/ts/wasm native software it would not only be very heavy to load the code(typescript compiler alone is >20mb) for each app's runtime, but it would also be potentially be insecure to give the required permissions(like file system read/write) to said runtimes.

I also already have a specific piece of infrastructure that would benefit from this system in production(other than debugging/integration tools).
One of the systems I've been working on for my projects, a distributed immutable event storage/query service, would potentially require a lot of memory(>256MB) to cache indexes, since my goal is to be able to perform queries directly from events instead of a view model(when possible). It would likely be more efficient if tenant apps could share caches. It would also be possible to perform preemptive fetching(and caching) of data need to satisfy predicted requests I.E.: If one app queries a user account it might go ahead and start the process to obtain and cache indexes for data related to said user.