Fragmented send (todo: compression)

[davidmurdoch]

Node can't create strings greater than about 1 GB, and Buffers max out at around 2GB. Because of this limitation I've broken up the message into small chunks. I can then send each chunk over the http server like this:

response.cork(() => {
  // ... I've omitted the header write steps for brevity

  for (const chunk of chunks) response.write(chunk);
  response.end();
});

And this works beautifully.

I've tried to find a way to do this using the ws api but send seems to require the whole message. Is there a way to stream chunks into send, or maybe send individual websocket message fragments from JavaScript?

_ observation: even if I can solve this issue the receiving end would need to be able to handle the large ArrayBuffer it will ultimately receive in the single message event - but that is a problem for another day. 😅 `

[ghost]

I've never heard of that issue but fragmented sends should be simple to add:

ws.sendFragment('lalalala', uWS.FIRST);
ws.sendFragment('lalala', uWS.CONTINUATION);
ws.sendFragment('lalla, uWS.FIN');

Then you can also use proper backpressure (onDrain) to throttle sending. Something like this should be simple to add.

[davidmurdoch]

An interface like that would be perfect. I'm guessing I'd have to manage a message queue locally so as not to interleave websocket frames for different messages.

My use case is generating and sending a large program trace, the largest I've seen is just over 10gb, though 1-2 GB traces are much more common. These traces are generated on the fly, so throttling the program trace generation via backpressure would work especially well here.

[ghost]

Yes you would have to keep track of things. I could make it more automatic but I don't want to because nobody has raised this issue and implementing this, simpler, interface is super simple and comes with no overhead to others.

[ghost]

Btw, the talk we had on Electron could make sense now that we link to BoringSSL and don't depend on the (non existing) OpenSSL of Electron.

[davidmurdoch]

Yes you would have to keep track of things.

👍 Yeah, it makes sense to keep such an edge case like this as simple as can be.

Electron could make sense now that we link to BoringSSL

I noticed the work you did there and was planning on trying to get it working for us sometime in the next couple of months. I'll try to play around with it sooner rather than later!

[lpinca]

@davidmurdoch

I've tried to find a way to do this using the ws api but send seems to require the whole message.

Fragmented messages are supported in ws since forever, see https://github.com/websockets/ws/blob/master/doc/ws.md#websocketsenddata-options-callback

for (let i = 0; i < chunks.length; i++) {
  ws.send(chunks[i], { fin: i === chunks.length - 1 });
}

or from a Readable stream with backpressure handling.

(function read(readable) {
  const chunk = readable.read();

  if (chunk === null) {
    ws.send(Buffer.alloc(0), { fin: true });
    return;
  }

  ws.send(chunk, { fin: false }, function (err) {
    if (err) return handleError(err);

    read(readable);
  });
})(readable);

[e3dio]

Hey its the lead dev from the ws library :) I think he was referring to the uws library ws.send function tho

[davidmurdoch]

@lpinca I was referring to the uWS websocket API, as it also has an HTTP API.

We have a pure JS implementation of the uWS API (not a strict port, just the API) that uses the ws node library, so thanks for letting me know about the fragment feature; it will certainly be useful if and when uWS gets its own fragment feature!

[ghost]

I've added sendFirstFragment, sendFragment, sendLastFragment. Currently compression does not work so set it to false.

[ghost]

This is in v20.3.0

[uasan]

largest I've seen is just over 10gb, though 1-2 GB

It makes sense for your case implement io_uring
#1355