Enable ETAG based validation on every block download to provide higher consistency #1608
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vibhansa-msft
requested review from
souravgupta-msft,
ashruti-msft,
syeleti-msft and
jainakanksha-msft
as code owners
|
Later we should run some perf tests with data consistency on to inform users what % diff in performance, memory usage they can expect with this option on and off. |
ashruti-msft
approved these changes
Jan 20, 2025
jainakanksha-msft
approved these changes
Jan 20, 2025
…k with quotes while some apis dont:
| opt.Range = blob.HTTPRange{ | ||
| Offset: offset, | ||
| Count: len, | ||
| if etag != nil { |
There was a problem hiding this comment.
No need to check the nil value as we are passing the pointer to a zerovalue
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✅ What
In block-cache mode, every download is not validated for higher consistency. If a file is open or half-read and contents are modified on the container, subsequent reads will start fetching new content and provide inconsistent data to application. Adding an ETAG based validation will guard applications against such data corruption. When an 'open' file call comes ETAG from blob will be preserved by block-cache. Any subsequent block download will revalidate that ETAG is still the same. If ETAG has changed, then it means blob was modified on container after
openand in such case download and read operation will fail.🤔 Why
This will safeguard application from data corruption where blob is modified on container through other means.
👩🔬 How to validate if applicable
Open a file and then modify the file on container followed by a read operation on mounted path.
🔖 Related links
NA