After reading Socket Programming HOWTO — Python 3.10.7 documentation it became clear to me that doing socket communication right is something that has its challenges. And that most examples given on the internet do it wrong.
So I am wondering is there some library that works at a slightly higher abstraction level than the sockets package which could make it easier to do correct client-server data exchange between processes via sockets? Ideally the server would be implemented via threads or processes to handle more than one connection simultaneously.
All the libs that are higher level, I think, depend on you picking a specific protocol that you want to used.
Which protocol do you need to support?
1 Like
I have two scenarios:
-
there is one receiver and 1 ore more senders: The senders send messages (which can be made to be of fixed length if necessary) and the receiver has to read and act on the messages as quickly as possible.
-
this is like 1, but the receiver also has connections to a couple of workers. The receiver sends data to the worker, proceeds to do whatever else needs to be done and as soon as a worker sends back results, receive the results and use them for whatever else needs to be done. In that scenario, the receiver will also pass on data to another node, which should happen without blocking during the write.
Currently the simple scenario in 1 is more important, with regard to 2 I am still not sure about the details: since in scenario 2 everything is running on the same (linux) machine, i had been thinking that it might be better to exchange the actual data via shared memory and coordinate availability and successful copying via semaphores (another aim in addition to not doing it wrong is to be rather fast and responsive in both scenarios)
Thank you, this looks like a good suggestion! I was not able to really understand all the details so far, but from a first look this seems to work very well!
1 Like