cancelation of multishot requests

[romange]

Apologies if this has been discussed. I would like to understand how we cancel multishot requests.

1. Does closing a file descriptor (with a regular close() syscall) in the same thread where iouring operates
   will guarantee that no completion will be delivered afterwards?
2. Is there an opcode that cancels a multishot request, and if yes, -
3. can a multishot completion arrive after the cancelation completion has been received?

[axboe]

1. You can still have a completion racing with the close, so you should expect (at most) one more completion for this case. Either a successful one because it raced with close, or one that doesn't have IORING_CQE_F_MORE set as that should always terminate a multishot request.
2. Normal cancelation works regardless of the type.
3. Depends, the cancelation request status will tell you that. If it got canceled successfully, then the cancel cqe for the multishot will be ready in the CQ ring along with the cancelation request CQE. If it returns -EALREADY, then a completion could already be in the ring, or it could be on the way.

[romange]

@axboe FYI, verified that close(fd) does not send a final completion request with IORING_CQE_F_MORE reset.
Did it with a simple test, connecting a (client) socket to a server socket, and then sending on the server socket two packets and then closing the client socket. I got 2 completions corresponding to these writes with
flags=IORING_CQE_F_MORE | IORING_CQE_F_BUFFER and never got the third one. I was waiting for 3 seconds to be sure.
It's not a deal breaker cause I will use io_uring_register_sync_cancel with IORING_ASYNC_CANCEL_FD but maybe you want to check it out.

Kernel: 6.5.0

[DylanZA]

that's not totally unexpected. can you try shutdown() the socket first? that should trigger it to send a CQE.

this is similar behaviour to triggering a close() while doing a blocking recv(). The fd (integer) might go away, but internally there is still an open socket waiting for all outstanding requests to complete

[romange]

I will check. However, i wonder what's the expected timeout after which a final CQE will be sent in case the socket was closed abruptly.
I put a timeout for 3s and it did not arrive nor there any processing on this socket durign that time. In any case, it's not something that is blocking me but it may break the invariant stated below.

[axboe]

There's not necessarily any amount of time if you just close it, as there's still another reference to this file. This is why shutdown is needed, either shutdown(2) or through an io_uring equiv shutdown request, as Dylan stated.

[axboe]

Just to be clear, for any multishot request, you should always expect more completions as long as IORING_CQE_F_MORE is set, and never expect any further completions if IORING_CQE_F_MORE isn't set in the CQE.

[romange]

Got ya, so I need to make sure I free referenced resources after the last callback arrives, not matter how I trigger the cancelation.