a couple of fixes for problems in the code base

[rlehfeld]

Hi,

when using RPyC, I run into a couple of issues. Further, while working on the fix, I run into more issues discovered by the unit tests unrelated to my first changes. Currently all tests are green. I even executed all the tests more than 3000 times to verify everything working reliable (1500 non-bound and 1500 bound thread executions). So here comes the summary of things I detected and fixed.

1. when using pipes, the sending side might block when nobody is reading from the other side. This problem might happen when working with a pair of pipes especially where one is used for reading and one is used for writing. It might be, that both ends are at the same time writing. This problem is solved through moving the reading to a separate thread. I implemented this only for the unix variant as I am not aware if the same problem exists in the windows world

2. I detected that in quite some places threads are spawned and later on "joined". Actually the join will be ignored by python if the thread is a daemon thread. As all threads were spawned via the rpyc.lib.spawn this means no thread was actually joined. I fixed this through adding a new interface called worker which will be a non daemon thread and changed the usage at the places were there was a join. Further, I added some more thread cleanups where it was missing. Similar I did with spawn_waitready which was renamed to worker_waitready. All places within the rpyc extensions and the tests should now join with their spawned threads except one situation where we can't. Here I still use a daemon thread for joining with a cleanup thread. But when this situation arises, it means also, that the thread died already.

3. for bind_threads in the _cleanup, it might be that this is called from one of the worker threads from the thread pool. A worker thread cannot join with itself and this causes the cleanup to through an exception. I fixed this through a detection mechanism that we are in this situation and move the call in this case to a new thread which will not be part of the threadpool. I guess this solves also the original "todo where?" but kept the comment for the moment

4. for bound threads, the _serve_bounds inside protocol included several races leading to message not being handled or threads not terminating and thus to problems when using join. This problem was visible at least once around every 30 - 250 execution. Since I introduced these changes I executed more than 1500 test loops and no more issues were visible.

5. the python version given in the matrix of the github pipeline was not used. I adapted the typo and fixed the test_ssl.py tests accordingly. Further, with python 3.14.0 alpha flake8 is broken. I updated to the lastest version which are available on github which means in particular I switched to 3.13 final and 3.14rc1.

6. fixed the test_gdb.py tests. Gdb functionality must only be called from python from the main thread as otherwise it can and on my linux sytem will cause crashes of gdb. This could be easily solved by switching to OneShotServer. Apart from the, the teardown in the test included a wrong ordering of calls.

7. fixed a problem with gevent being loaded in tests apart from gevent test through moving the service to a separate process.

8. instead of using shutdown on a socket in RW direction, now we only do a shutdown in W direction and wait for the peer to detect the shutdown and close or using shutdown in it's own W direction. This solves also the problem with test_ssl.py as now the peer can receive the reason of a failed shutdown reliable.

9. when calculating timeout, monotonic time should be used in order to be not affected by time deviations e.g. caused by NTP.

10. as the SSL certificates used for testing expired, I recreated them so that the SSL tests run through again. A proper fix would be to move this into a github pipeline itself so that these certificats are always "fresh"

11. there is a race in closing and polling threads which leads to an error. This race is solved as well. (actually, this is the thing I wanted to fix in the first place)

Best Regards

PS: the fails in the Spinx test all look like false positives but I could not find out how to handle this

René