File seek from end

[davidzeng0]

I'm implementing a seek function that allows the user to seek in the same way as lseek, and came up with two possible implementations.

1. Run an async statx for the file length and then set the offset variable
2. Run an lseek synchronously and then proceed to use that as the offset. Though I'm a little worried as it seems like it might block

Edit: hmm seems like statx only takes file paths, which may be a little inconvenient

My program is entirely single threaded so there is no worry for synchronization.
What would you say is the best approach? Thanks.

Maybe a feature can be added to io_uring as SEEK_END -> -1 - (bytes_from_end + 1) ? It seems like the kernel uses signed long internally anyway, even though the read/write functions use u64

[davidzeng0]

For now just running an async statx on file open, but would love to see some seeking APIs in io_uring :)

[axboe]

Generally using the file position for anything doesn't make a lot of sense for an async API, where you can have multiple IOs inflight. There are obviously cases where it can make sense, depending on the use case, I'm just speaking generically here. This is why I'd be hesitant to add an lseek variant.

Are you just looking for end-of-file? This is obviously also potentially racy depending on the use case, but I can see that making sense to expose. We'd just need to add an opcode for it. Ideally I'd like the stat variants to be sync-issue capable, in which case we could just use those.

[davidzeng0]

I imagine the general usecases being

1. highly parallel disk read/writes
2. (my usecase) having a uniform async interface to turn synchronous APIs into async, which would enable building a web server, for example.
3. maybe some others, don't have it off the top of my head atm

I did a little bit of digging in the kernel code, but I couldn't find where the file ops for open were assigned, so I just assumed lseek is blocking

-- some backstory --
A few years ago, I used epoll, and had to spawn up some worker threads to do file reads. I found out about io_uring, and I immediately switched over, but still kept the threadpool as I wasn't aware of openat being the same as open. Realizing that, I got rid of the threadpool, until I needed to getaddrinfo, so I added it back. Then I found out about c-ares, an async getaddrinfo so I axed the threadpool again. But basically it boils down to: I'd prefer to not have a threadpool if lseek can be done efficiently in the kernel, or if it ever gets added in the future, because then I'll probably have to axe the threadpool again.
-- end of backstory --

I do agree with you in that adding a seek op code doesn't feel like that great of an idea as it feels like it doesn't really belong with the other op codes.

I've thought about the various ways one could arrive at a file end position in an io_uring based application:

1. run lseek on a threadpool
2. using the negative numbers in off_t as a seek-backwards offset
3. store the filename (which isn't great) and run an async statx before a seek
4. run the statx before file open (assuming the file is never written to)
5. some way to query a file's position and length without blocking (URING_CMD?)

I think # 2 wouldn't be too strange to implement, as setting offset=-1 (#32) faces the same concurrency issues and adding negative offsets can be easily integrated without any changes to the existing API. Unfortunately, the user can no longer keep track of the offset (unless maybe with a NOTIF cqe?)

For now, my basic usecases suffice with option # 4. But if async seeking or any of the options above could be reasonably implemented in a way that makes sense, it would be amazing.