Default/mutable buffer uploading

[RobertMtx]

For data that may change once every 1-100+ frames, what would you recommend as the GPU uploading scheme? Should small data objects generally be uploaded in a single call, possibly wasting bandwidth? Or is it, in general, better to optimize the amount of data uploaded by increasing the number of uploads? IE, skipping over unchanged data?

I'm currently planning to upload using context::UpdateBuffer(), but I'm not sure how to partition the data. I can either iterate through the array of objects and upload those that have changed that frame, one at a time. Or figure out the beginning and end points of the changes and upload everything at once, even if that means uploading stuff in the middle that hasn't changed. For the objects in question, there are likely to be 50 or so of them, but only 1-5 changing at any given time.

And there's also a question of how this should be handled when those objects don't reside in adjacent memory. Would it be worth it to copy all of them to another chunk of memory so they are adjacent, so it can be copied in one call? Or would it be just as effective to upload them one at a time?

I will definitely test some of these things at some point, but I'm curious to hear from someone that knows much more about these things. Even when things test well, I'm skeptical to rely on it, due to lack of exposure to other cards/PCs and a general lack of past experience with graphics programming. I appreciate any info you have time for.

[TheMostDiligent]

Should small data objects generally be uploaded in a single call, possibly wasting bandwidth?

The general rule of thumb in GPU programming is to batch commands as much as possible. Updating all at once is going to be much more efficient than issuing many commands to update small parts.

Or is it, in general, better to optimize the amount of data uploaded by increasing the number of uploads? IE, skipping over unchanged data?

That largely depends on your specific use case. If you need to re-upload megabytes of data just to update a few bytes, that is probably not going to be optimal. If you need to update, say at least 50% of the data, then batching will be benefcial.

I can either iterate through the array of objects and upload those that have changed that frame, one at a time. Or figure out the beginning and end points of the changes and upload everything at once, even if that means uploading stuff in the middle that hasn't changed.

Depends on how much data you need to update. UpdateBuffer is much more expensive than mapping a dynamic buffer. If you update few KBs of data, just do it.

And there's also a question of how this should be handled when those objects don't reside in adjacent memory. Would it be worth it to copy all of them to another chunk of memory so they are adjacent, so it can be copied in one call?

Unless your object data is really tiny (like, few bytes), this likely will not give much benefit. But I don't know really.

Also, always measure your performance and don't do premature or speculative optimizations - optimize only what is know to be the bottleneck from profiler.

[RobertMtx]

Really appreciate your advice, thanks! I had a feeling that batching the uploads would usually be the way to go. Most of my updates actually are through dynamic buffers. But in this case, the data is not likely to change every frame. It can vary, but in the most common scenario, they may not update at all, or with quite some time between the updates. Thanks!

[TheMostDiligent]

In Direct3D12 and Vulkan, mapping dynamic buffers is fast and uploading data to them is the cost of memcpy. However, amount of dynamic memory is limited and they may be a bit slower on GPU. For small data that is uploaded frequently, dynamic buffers are optimal.