Use a single "message" event listener to dispatch received messages #653

achim-k · 2024-01-18T22:14:37Z

From #649:

We make heavy use of Comlink for our WebWorker communication. On some recent profiling traces I noticed a hot-path on the "addEventListener" line.

While its hard to say exactly how the event listeners are stored, most implementations I've seen for such interfaces involve storing them in an array and removing the listener equally requires adjusting an array. The existing logic also creates two closures - one for the Promise (which is unavoidable from what I can tell), and another for the event listener. As it was suggested in #647 there is a potential for improvement by using a single "message" event listener and doing the dispatch manually by looking up the resolve functions in a Map.

This definitely reduces the runtime cost of the requestResponseMessage call itself - though there's still cost to new Promise and the closure. There's a shifted cost to the ID lookup on the map though in my profiling that did not show up as a hot path item.

Conceptually using a map for the lookup and one handler seems like it should perform better but if folks have some ideas on how to more robustly benchmark this PR that would be helpful.

This PR is similar to #649 and #651 but avoids that references to the resolve functions or the endpoint are kept in memory.

The performance impact of this PR gets more noticeable the more parallel requests are made:

See also #651 (comment)

Co-authored-by: Roman Shtylman <[email protected]>

google-cla · 2024-01-18T22:14:41Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

defunctzombie · 2024-01-19T16:38:33Z

src/comlink.ts

+      return;
+    }
+    const resolver = pendingListeners.get(data.id);
+    if (resolver) {


nit: flip this around to early return so that early return is the "no more work to do case"

if (!resolver) { return; } // actual work ...

defunctzombie · 2024-01-19T16:39:16Z

src/comlink.ts

+    }
+  });
+
+  return createProxy<T>(ep, pendingListeners, [], target) as any;


Why does this case as any? I know this was also true of the earlier code but in our code (foxglove) we always avoid any - are we not able to get the correct type here?

Not sure. But seems like it's not that easy, quoting the README:

Comlink does provide TypeScript types. When you expose() something of type T, the corresponding wrap() call will return something of type Comlink.Remote<T>. While this type has been battle-tested over some time now, it is implemented on a best-effort basis. There are some nuances that are incredibly hard if not impossible to encode correctly in TypeScript’s type system. It may sometimes be necessary to force a certain type using as unknown as <type>.

defunctzombie · 2024-01-19T16:42:04Z

src/comlink.ts

-      ep.removeEventListener("message", l as any);
-      resolve(ev.data);
-    } as any);
+    pendingListeners.set(id, resolve);
    if (ep.start) {
      ep.start();
    }
    ep.postMessage({ id, ...msg }, transfers);


Random thought for the future - but it would be nice to avoid this spread operator - I don't see any reason for it when we could instead have id, payload or similar. While not a massive perf issue, removing it is fewer work cycles for the runtime with no downside to the interface or logic since this is an internal structure.

Could be something for a separate PR?

MikalDev · 2024-03-02T03:09:04Z

This change had a huge impact on my work with a physics library. I am calling a function on main thread to add physics bodies in the worker thread. Originally it was a huge hot path, because I was stress testing creating 5000 bodies in one 16ms tick.

I thought I would have to instead batch up the commands and parameters and send over in one message (which still might be a good idea), instead I can now continue to use the comlink proxy for the function. Nice work.

I am now using the foxglove fork for my project to pick up the other nice to haves like faster id generation.

lvivski · 2024-03-26T18:29:52Z

@achim-k Thank you for this change! I think the performance improvements are great. Do you still need to follow up on something or at this point are just waiting for the final PR approval?

achim-k · 2024-03-26T18:59:21Z

@achim-k Thank you for this change! I think the performance improvements are great. Do you still need to follow up on something or at this point are just waiting for the final PR approval?

This is just waiting for the final approval. We are already using this in production for a couple of months, without issues.

lvivski · 2024-03-26T19:56:40Z

@surma is this PR still interesting from the maintainer's perspective? I understand that you might have different opinions on how this should be implemented. I think this is a useful change for high-throughput applications.

ivancuric · 2024-06-10T07:59:39Z

Bump

ThaUnknown · 2024-09-16T15:02:04Z

is there any ETA for this? with this comlink could be adapted for use with generic message channels, such as websocket, text, etc and no longer be bound to just web workers

surma

This looks good to me. Thanks so much.

surma · 2024-11-07T12:34:43Z

published in v4.4.2

lvivski · 2024-11-07T17:47:19Z

Thank you!

ThaUnknown · 2025-01-02T20:08:33Z

src/comlink.ts

@@ -402,7 +425,7 @@ function throwIfProxyReleased(isReleased: boolean) {
 }

 function releaseEndpoint(ep: Endpoint) {
-  return requestResponseMessage(ep, {
+  return requestResponseMessage(ep, new Map(), {


Why did noone ever address this? this now leaks INDEFINETLY, causing ep[finalizer]() to never be called!!! this means in browsers you will leak threads if you had worker.terminate linked to the exposed [finalizer] property, this promise will never resolve!!!

Which part are you saying is leaking?

doing

class SelfTerminate extends Worker { [finalize](){ this.terminate() // never fires } } const x = wrap(new SelfTerminate('./w')) x[releaseProxy]()

means the worker never terminates.... it used to before this change, thats because the releaseEndpoint promise never settles

Hm, I don't understand how this worked before. When releasing the proxy, the finalizer of the exposed object (the object created in the worker) is called, not the finalizer of the SelfTerminate instance. I have tested this with v4.4.1 which does not include this change.

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Comlink Worker Example</title> </head> <body> <script type="module"> import * as Comlink from 'https://cdn.jsdelivr.net/npm/[email protected]/dist/esm/comlink.mjs'; // Inline worker definition const workerCode = ` importScripts('https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/comlink.js'); const workerAPI = { add(a, b) { return a + b; }, // Uncomment function below to have worker close itself when proxy is released. // [Comlink.finalizer]() { // console.warn("Closing worker..."); // self.close(); // } }; Comlink.expose(workerAPI); `; class SelfTerminate extends Worker { [Comlink.finalizer]() { console.warn("Terminating worker..."); this.terminate(); } } const blob = new Blob([workerCode], { type: 'application/javascript' }); const worker = new SelfTerminate(URL.createObjectURL(blob)); (async function() { const workerAPI = Comlink.wrap(worker); const result = await workerAPI.add(5, 7); console.log(`The result of the addition is: ${result}`); workerAPI[Comlink.releaseProxy](); })(); </script> </body> </html>

ah, sorry, a minor missunderstanding on my part, I wrote this when tired, it needs to be a message port, which then is never closed https://github.com/GoogleChromeLabs/comlink/blob/main/src/comlink.ts#L395

OK got it. I'll submit a PR that fixes this.

await releaseEndpoint(ep, pendingListeners) function releaseEndpoint (ep: Endpoint, listeners: PendingListenersMap = new Map()) { return requestResponseMessage(ep, listeners, { type: MessageType.RELEASE }).then(() => { if (finalizer in ep && typeof ep[finalizer] === 'function') { ep[finalizer]() } }) }

is how i fixed it on my end

Could you verify if the change in foxglove#6 also works for you? If yes, then I'll submit a PR to this repo here.

Use a single "message" event listener to dispatch received messages

af0ba68

Co-authored-by: Roman Shtylman <[email protected]>

defunctzombie reviewed Jan 19, 2024

View reviewed changes

defunctzombie mentioned this pull request Jan 20, 2024

Use a single "message" event listener to dispatch received messages #649

Closed

address comments

f987ffb

This was referenced Jan 24, 2024

Use a single "message" event listener to dispatch received messages foxglove/comlink#2

Closed

Use a single "message" event listener to dispatch received messages foxglove/comlink#3

Merged

surma approved these changes Nov 7, 2024

View reviewed changes

surma merged commit ad4d2d1 into GoogleChromeLabs:main Nov 7, 2024
1 check passed

ThaUnknown reviewed Jan 2, 2025

View reviewed changes

astrolox mentioned this pull request Jan 11, 2025

Significant performance optimizations possible in requestResponseMessage #647

Open

achim-k mentioned this pull request Jan 13, 2025

Fix message ports not being closed when proxy is relased #678

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use a single "message" event listener to dispatch received messages #653

Use a single "message" event listener to dispatch received messages #653

achim-k commented Jan 18, 2024 •

edited

Loading

google-cla bot commented Jan 18, 2024

defunctzombie Jan 19, 2024

defunctzombie Jan 19, 2024

achim-k Jan 24, 2024

defunctzombie Jan 19, 2024

achim-k Jan 24, 2024

MikalDev commented Mar 2, 2024 •

edited

Loading

lvivski commented Mar 26, 2024

achim-k commented Mar 26, 2024

lvivski commented Mar 26, 2024

ivancuric commented Jun 10, 2024

ThaUnknown commented Sep 16, 2024

surma left a comment

surma commented Nov 7, 2024

lvivski commented Nov 7, 2024

ThaUnknown Jan 2, 2025

defunctzombie Jan 2, 2025

ThaUnknown Jan 2, 2025

achim-k Jan 3, 2025

ThaUnknown Jan 3, 2025

achim-k Jan 6, 2025

ThaUnknown Jan 6, 2025

achim-k Jan 7, 2025

Use a single "message" event listener to dispatch received messages #653

Use a single "message" event listener to dispatch received messages #653

Conversation

achim-k commented Jan 18, 2024 • edited Loading

google-cla bot commented Jan 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MikalDev commented Mar 2, 2024 • edited Loading

lvivski commented Mar 26, 2024

achim-k commented Mar 26, 2024

lvivski commented Mar 26, 2024

ivancuric commented Jun 10, 2024

ThaUnknown commented Sep 16, 2024

surma left a comment

Choose a reason for hiding this comment

surma commented Nov 7, 2024

lvivski commented Nov 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

achim-k commented Jan 18, 2024 •

edited

Loading

MikalDev commented Mar 2, 2024 •

edited

Loading