Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema Evolution & Compatability & Whatnot #24676

Draft
wants to merge 11 commits into
base: dev
Choose a base branch
from

Conversation

oleiman
Copy link
Member

@oleiman oleiman commented Jan 3, 2025

.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

@oleiman oleiman self-assigned this Jan 3, 2025
@oleiman oleiman force-pushed the dlib/core-8525/compat-visitor branch 2 times, most recently from 961b6dc to 2052d15 Compare January 6, 2025 02:37
variant:
  - bool is_new
  - bool removed
  - struct src_info
    - id of the source field
    - requiredness of the source field
    - optionally source field type (if primitive)

Used in subsequent commits to:
  - cheaply identify struct fields that have changed shape
  - defer type & requiredness invariant checks to the end

Only track primitive types because
a) they are cheap to copy and
b) structured types are effectively checked during traversal
Templatize the visitor to add a const nested_field* version.
@oleiman oleiman force-pushed the dlib/core-8525/compat-visitor branch from 2052d15 to b2d3ba0 Compare January 6, 2025 07:13
And lightly refactor test suite base class for reuse
Make a copy of the destination schema.
Generate a schema transform from source to dest.
Apply the transform to the copy and return the result (or an error code)
Generates increasingly nested structs and measures the time
it takes to:

- collect all fields of two copies w/ field_collecting_visitor
- generate a schema transform from one copy to another
- visit a generated schema transform

Serves as a decent point of comparison between the limited schema
verification already present in the tree and the utilities introduced
in this commit stack.
@oleiman oleiman force-pushed the dlib/core-8525/compat-visitor branch from b2d3ba0 to a845692 Compare January 6, 2025 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant