feat: new hooks API

[ronag]

While this is not breaking per se. It's pretty much unusable with existing handlers...

[ShogunPanda]

I think you're not passing a controller here.
Also, in the onRequestStart above, you don't pass it so I would remove the comment at all.

[ronag]

So we have 2 different controllers, one for the request and one for the response, they may run in parallel.

[ShogunPanda]

One nit, rest LGTM!

[metcoder95]

LGTM, nice work; I can see the potential simplification of coordinating the flow control through the hooks thanks to the controller.

Just have some open questions left.

[metcoder95]

Curious, how are we planning for the hook to hint the controller about backpressure, as it seems it does not receives the controller?

[ronag]

I was thinking the user saves a reference to the controller from the start hook. By I guess we could always send in the controller?

[metcoder95]

That's fair, we can keep it simple and keep the controller for that

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 90.38462% with 10 lines in your changes are missing coverage. Please review.

Project coverage is 93.50%. Comparing base (cae5625) to head (d1c5e47).
Report is 5 commits behind head on main.

Files	Patch %	Lines
lib/core/request.js	89.58%	10 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3054      +/-   ##
==========================================
- Coverage   93.51%   93.50%   -0.02%     
==========================================
  Files          89       89              
  Lines       24331    24408      +77     
==========================================
+ Hits        22754    22822      +68     
- Misses       1577     1586       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ronag]

@mcollina @ShogunPanda @metcoder95

• Should we have separate controllers for request and response or just one for both or just the response? In theory the request and response can propagate independently hence it might make sense to be able to pause/resume them separately, on the other hand it's not a usual case and maybe just adds unnecessary complexity?
• Should it be possible to pause/resume the request? Or should we always leave it to the server to apply back-pressure? One use case would to be able to throttle the client upload.

[ShogunPanda]

@mcollina @ShogunPanda @metcoder95

• Should we have separate controllers for request and response or just one for both or just the response? In theory the request and response can propagate independently hence it might make sense to be able to pause/resume them separately, on the other hand it's not a usual case and maybe just adds unnecessary complexity?

I would keep one controller which acts on both. pause, resume and abort can act on both the request and response and the client can only be in either "sending request" or "receiving response" state at a time so no room for ambiguity.

• Should it be possible to pause/resume the request? Or should we always leave it to the server to apply back-pressure? One use case would to be able to throttle the client upload.

I definitely would allow to also control the request flow.

[metcoder95]

@mcollina @ShogunPanda @metcoder95

• Should we have separate controllers for request and response or just one for both or just the response? In theory the request and response can propagate independently hence it might make sense to be able to pause/resume them separately, on the other hand it's not a usual case and maybe just adds unnecessary complexity?

I would keep one controller which acts on both. pause, resume and abort can act on both the request and response and the client can only be in either "sending request" or "receiving response" state at a time so no room for ambiguity.

Agree with @ShogunPanda towards a single controller, though I believe that most of the time implementers will look to control the stream flow over the response rather than the request; by providing a controller that allows them to also alter the flow of the request we might enable scenarios where users puts themselves into a weird spot and undici not being fully able to do its work accordingly.

The last being said, I'd prefer to keep the controller only for the response and we can evaluate later if there's a need for one over the response.

• Should it be possible to pause/resume the request? Or should we always leave it to the server to apply back-pressure? One use case would to be able to throttle the client upload.

I definitely would allow to also control the request flow.

Couldn't we achieve that already with Dispatch#compose?
They can intercept the dispatch and throttle the body as they need

[ronag]

the client can only be in either "sending request" or "receiving response" state at a time so no room for ambiguity.

Actually, you can be sending and receiving at the same time. Though I'm not sure how common that is.

Couldn't we achieve that already with Dispatch#compose? They can intercept the dispatch and throttle the body as they need

Yes, that's also possible. Though less "nice".

@metcoder95 @ShogunPanda So in theory we could totally skip the request hooks (at least for now). However, there is one problem. We need to be able to abort the request before the response has been received (onResponseStart). That's the only place where we actually need to interact with the request.

i.e.

onRequestStart(controller: { abort: Function })

But I guess we can just implement that and leave the rest for the future. Or can you think of a nicer way? Preferably without any onRequestXXX hook.

One alternative is to keep onConnect (I'd prefer not).

[ShogunPanda]

the client can only be in either "sending request" or "receiving response" state at a time so no room for ambiguity.

Actually, you can be sending and receiving at the same time. Though I'm not sure how common that is.

Gosh, you are right. Why do I always forget about pipelining? XD

Couldn't we achieve that already with Dispatch#compose? They can intercept the dispatch and throttle the body as they need

Yes, that's also possible. Though less "nice".

@metcoder95 @ShogunPanda So in theory we could totally skip the request hooks (at least for now). However, there is one problem. We need to be able to abort the request before the response has been received (onResponseStart). That's the only place where we actually need to interact with the request.

i.e.

onRequestStart(controller: { abort: Function })

But I guess we can just implement that and leave the rest for the future. Or can you think of a nicer way? Preferably without any onRequestXXX hook.

One alternative is to keep onConnect (I'd prefer not).

Let's skip it for now. I would just say let's code it in a way we can introduce it later in a semver-minor.

[metcoder95]

@metcoder95 @ShogunPanda So in theory we could totally skip the >request hooks (at least for now). However, there is one problem. We >need to be able to abort the request before the response has been >received (onResponseStart). That's the only place where we actually >need to interact with the request.

i.e.

onRequestStart(controller: { abort: Function })

But I guess we can just implement that and leave the rest for the future.

Agree and in sync with @ShogunPanda, let's do that 👍