Server is going down

[maybeshaed]

Hello everyone, I ran into a problem using Ratchet as a websocket on a project. The project has clients in the form of users, as well as in the form of units with which users interact through the Central Server, all this communication is implemented using a websocket.

So, the problem: the server completely randomly starts to fall off (namely, the units with which users interact), the web socket lags and connecting to it via websocat takes 25+ seconds, which is a problem for units in getting a handshake with the server. Also, our database subsequently begins to be overloaded due to the large number of requests (I’m not sure that the websocket has anything to do with this, but rather my poor database setup). Also, one of the interesting phenomena is that after a drop, units begin to behave strangely, one drops, another connects, and so on in a circle, the only thing that helps us is rebooting the entire server, and even this does not always work, we have to reboot twice. When these problems are absent, if you reboot the machine, the units are connected back in a minute (about 100 units), and during the fall it doesn’t even take us 15 minutes to get everything back up. Below I attach a screenshot of what it looks like during the “fall”.

Thanks in advance everyone for the answers.

Info:
CentOS 7
php 7.4
mysql

[danilashtefan]

I have the very same issue in production. I am experiencing it for over 6 months. After some random amount of time server performance degradation is noticeable with exactly same description. After throughly verifying my DB setup and server parameters during this performance drop, I can confirm that the issue is in the web socket layer itself. Another noticeable fact is that it happens randomly, not even on the peal load.

I have found similar issues with people experiencing same. For instance #328. But there is not solution provided, just assumptions.

I really hope that contributors will react on this thread and give some advices. All these is making use of Ratchet really frustrating...

[newmediacrew]

Have you tried a simple service deamon. We are using this for years now and no issues whatever. This is running on unbuntu 22.04 lts

[Unit]
#service description
Description=The ratchet socket server
#start at level 3, after booting the network target
After=network.target

[Service]
Type=simple
#what do we want to start
ExecStart=/usr/bin/php /path/to/socketserver/SocketServer.php
TimeoutStartSec=0
#always restart on failure
Restart=always
#to avoid to much stress on the server only restart every 5 seconds
RestartSec=5

[Install]
WantedBy=multi-user.target