Experimental usrsctp support #1637
Conversation
Early stage of usrsctp (userspace sctp stack) support, clean up in code is needed. -Sending and receiving sctp data with gen_sctp in active and passive mode -peeloff association -basic setopts added
Data could be received between peeloff and that we set the notif fd to be notified on. Write some data on fd to force an check if we missed anything. Future fix, add passing notif fd to peeloff fd.
use event callbacks instead of fd in select to schedule port. On an event schedule the affected port directly as what select should have done when socket are ready. But since we don't have any real socket fd, there is no point in doing the round through a syscall to schedule the port.
Add support for passing already open rawsockets to usrsctp_init. Sockets are opened and fds are passed via arguments to the emulator.
AF_ROUTE and IPv6 raw sockets are used by usrsctp stack.
Active once uses a mutext to be able to keep track. But INET_ONCE wasn't reset to PASSIVE if we scheduled the port due to readable.
Since the support for opening rawsockets and passing the fds as arguments to emulator. There is no need to have the whole beam as setuid root on mac os anymore.
|
We have some plans to make big changes to how sockets are handled. I am not sure how this would fit in to this new scenario. We will have to ask you to be patient as this needs to be discussed further internally before we can give you good feedback. |
IngelaAndin
added
team:VM
|
Thanks for the update. |
|
The way that we normally solve the problem when a library does not expose an fd to do the polling but uses a callback, is to create a pipe that the driver can select on and then the callback can signal on that pipe. You should do that instead of adding functionality to the emulator. We still have not decided what we will do with this functionality yet, just wanted to leave the above feedback for the record. |
|
I had the approach of using a pipe that was put into the poll/select, and then I wrote to that pipe when I got a callback that something was on the socket. |
|
Does it also potentially enable webrtc data channel for erlang sctp module? |
|
@mko-io this only adds usrsctp as a different backend to provide general SCTP protocol features to erlang via the gen_sctp module. |
|
Also interested in usermode SCTP. Where is the afore-mentioned discussion taking place? |
|
We have a work package where we are working to replace the existing inet-driver with a nif-based solution. However this is big job and we do not want to make big changes to the existing inet driver, like this one, in the midst of this. We would rather evaluate this functionality in the light of the new solution. Alas this means that it might be stalled for quite a long time. |
|
@IngelaAndin I'm a bit of exciting about the driver to Nif conversion plan, so it worths waiting and hope the OTP team can add this feature afterward. |
|
@IngelaAndin I'm also looking forward to inet-driver implemented as a nif instead. |
|
any update on this? WebRTC Datachannel is enabled on all the browsers link and a better solution compares to WebSockets, and fit well for the process/messaging model for Erlang/OTP by enabling clients send message directly to each other. |
|
Hi @falkevik , I am just wondering if you have done something more with usrsctp after this PR? It could be interesting to explore the possibilities to use usrsctp together via our new socket API. |
|
Hi @KennethL I have not done any real work in porting / moving the feature over to the new socket API I'm afraid. |
|
Is there up to date documentation somewhere? E.g |
|
Have you tried to get any of your |
|
Thanks for having a look at the PR @RaimoNiskanen The main missing part is the event callback handling. The PR-SCTP compatibility addition added in the INIT handshake isn't critical. Some changes was related to providing pre-opened sockets for raw sockets. I.e. passing file descriptor to be used. I'm on vacation right now, so don't have that much time to look at the documentation you mention. |
|
Hi @falkevik have you looked any more at this? Do I understand correctly that if there is an Erlang VM with usrsctp support active it is only that Linux process that can receive (and send) sctp from the outside (other hosts)? |
Experimental support for usrsctplib. Any feedback appreciated.
The main idea is to use usrsctplib from within the emulator using
gen_sctpas for the current SCTP support.But this makes the SCTP stack run within the beam.
This mainly enabled me to have SCTP on macOS without running unsigned sctp kernel extensions.
It might also be possible to get usrsctplib support to work on windows.
I have only tested on macOS and Linux.
Some modifications has been made to usrsctp which can be found on below github repo and branch. I.e. passing already opened raw sockets to the lib and event callback support when socket is ready.
https://github.com/falkevik/usrsctp/tree/event_callback_support
So how to try it.
First install the usrsctplib from the above branch.
When done you can enable usrsctp support when configuring otp.
./configure --enable-sctp=usrsctpshould start looking for the usrsctp header.When done and compiled as usual.
On Linux you can go with adding cap_net_raw;
setcap cap_net_raw+ep /path/to/beam.smp.On macOS I pass already opened raw sockets as arguments to the beam.
New flags are
+zsctp_raw_ipv4 <fd>+zsctp_raw_ipv6 <fd>+zsctp_raw_route <fd>setuid_socket_warp can be used for this, which is slightly modified to set additional options on the raw socket before dropping privileges.
setuid_socket_wrap -t +zsctp_raw_ipv4,sctp -T +zsctp_raw_ipv6,sctp -z +zsctp_raw_route,sctpSo to the actual code changes, the idea is that when not enabled it should not affect at all.
This need extra code review I think, there a lot of
#ifdefs that might have ended up badly.There are some other changes that I need feedback on how to do it properly.
Since we don't have any real file descriptor that we can put in the
select/pollI register a callback when socket is ready to be read.When this callback is invoked, I check if the socket are in active mode or not.
If it is in active mode, the port are scheduled to be run
erts_port_task_scheduleas select would have done if the fd was ready.To be able to use this call from the inet_drv driver I had to make them available from somewhere.
Currently in
erts/emulator/sys/common/erl_sys_common_misc.c, where to put this kind of code?