Skip to content

Commit

Permalink
feat: Branch Key Storage Changes for Mutations
Browse files Browse the repository at this point in the history
  • Loading branch information
texastony committed Jan 18, 2025
1 parent 8d6bb56 commit 797a657
Show file tree
Hide file tree
Showing 2 changed files with 290 additions and 1 deletion.
157 changes: 157 additions & 0 deletions changes/2025-01-17_key-store-storage-mutations/background.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved."
[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0"

# Branch Key Storage Changes for Mutating Branch Keys

# Definitions

## All items of a Branch Key

"All items of a Branch Key" is all the items
persisted in the Key Store with the same Branch Key ID;
these will be one or more DECRYPT_ONLY versions,
the ACTIVE version, and the Beacon Key.

In this document,
the result of the kms:GenerateDataKeyWithoutPlaintext of a Branch Key Version
is the "Branch Key Version's Cryptographic Material".

All the results of the kms:GenerateDataKeyWithoutPlaintext
of all items of a Branch Key
are the "Branch Key's Cryptographic Materials".

## MPL Consumer

An "MPL Consumer" is a library, service, or other application
that uses the AWS Cryptographic Material Providers Library (MPL)
to manage cryptographic materials.
An "MPL Consumer" may or may not be using an AWS Crypto Tools product,
such as the AWS Encryption SDK or the AWS Database Encryption SDK.

## Conventions used in this document

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119).

# Background

The [branch key store](../../framework/branch-key-store.md)
persists Branch Keys and their metadata;
the [Key Storage Interface](../../framework/key-store/key-storage.md)
is the interface for persisting this data.

The Interface does yet specify how to safely change
properties of these Branch Key items.

This change adds new Operations to the interface,
which facilitate changing the
properties of all items of a Branch Key.

These new operations are constructed with _Optimistic Locks_,
ensuring that modifications do not accidentally erase data.

Additionally,
they facilitate the creation, maintenance, and deletion
of items that are used to ensure a change to a Branch Key
is consistently applied to all items of that Branch Key.

# Design Questions

These are not answered,
as the Mutations Background document addresses
these questions.

# Changes

## New Structures

The following structures are introduced to the Key Storage Interface:

### EncryptedHierarchicalKeys

A List of
[EncryptedHierarchicalKey](../../framework/key-store/key-storage.md#encryptedhierarchicalkey).

### MutationCommitment

A structure holding information on
an in-flight Mutation of a Branch Key
that is never updated in-flight.

The members are:

- `Identifier`: a string, The Branch Key (ID) under Mutation
- `CreateTime`: a string, The ISO 8061 UTC Timestamp of when the Mutation was started
- `UUID`: a string, A unique identifier for the Mutation
- `Original`: a binary blob, A commitment of the original mutable properties of the Branch Key
- `Terminal`: a binary blob, A commitment of the terminal mutable properties of the Branch Key
- `Input`: a binary blob, A Description of the input to initialize a Mutation
- `CiphertextBlob`: a binary blob, which MAY be used to protect the Mutation Commitment

### MutationIndex

A structure holding information on
an in-flight Mutation of a Branch Key
that is regularly changed in-flight.

The members are:

- `Identifier`: a string, The Branch Key (ID) under Mutation
- `CreateTime`: a string, The ISO 8061 UTC Timestamp of when the Mutation was started
- `UUID`: a string, A unique identifier for the Mutation
- `PageIndex`: a binary blob, Indirectly describes which items of the Branch Key have already been Mutated
- `CiphertextBlob`: a binary blob, which MAY be used to protect the Mutation Index

### OverWriteMutationIndex

A structure, much like
[OverWriteEncryptedHierarchicalKey](../../framework/key-store/key-storage.md#overwriteencryptedhierarchicalkey),
that affords a conditioned update to the Mutation Index.

The members are:

- `Index`: The new Mutation Index
- `Old`: The last read Mutation Index

## New Operations

The following new Operations are added:

### GetItemsForInitializeMutation

Retrieves the items necessary to initialize a Mutation,
while checking for any in-flight Mutations.

### WriteInitializeMutation

Atomically updates the Active Items
of a Branch Key along with
the Mutation Commitment and Mutation Index.

### WriteAtomicMutation

Atomically updates all the Items of a Branch Key.

#### QueryForVersions

Query Storage for a page of version (decrypt only) items of a Branch Key.

### WriteMutatedVersions

Atomically updates a page of version (decrypt only)
items while maintaining the Mutation Commitment and Mutation Index.

### GetMutation

Check for Mutation Commitment on a Branch Key ID.
If one exists, returns the Mutation Commitment.
Otherwise, returns nothing.

### DeleteMutation

Deletes an existing Mutation Commitment & Index.

#### WriteMutationIndex

Creates a Mutation Index, conditioned on the Mutation Commitment.
134 changes: 133 additions & 1 deletion framework/key-store/key-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@

## Version

0.2.0
0.3.0

### Changelog

- 0.3.0
- [Branch Key Storage Changes for Mutating Branch Keys](../../changes/2025-01-17_key-store-storage-mutations/background.md)
- 0.2.0
- [Mitigate Update Race in the Branch Key Store](../../changes/2025-01-16_key-store-mitigate-update-race/background.md)
- 0.1.0
Expand Down Expand Up @@ -74,6 +76,66 @@ A structure that holds two related [EncryptedHierarchicalKeys](#encryptedhierarc
- Item: the [EncryptedHierarchicalKey](#encryptedhierarchicalkey) that will be written
- Old: the [EncryptedHierarchicalKey](#encryptedhierarchicalkey) that was read and is presumed to be the currently persisted item that will be replaced by `Item`.

Both `Item` and `Old` MUST have the same Branch Key ID and Type.

### EncryptedHierarchicalKeys

A List of
[EncryptedHierarchicalKey](../../framework/key-store/key-storage.md#encryptedhierarchicalkey).

### MutationCommitment

A structure holding information on
an in-flight Mutation of a Branch Key
that is never updated in-flight.

The members are:

- `Identifier`: a string, The Branch Key (ID) under Mutation
- `CreateTime`: a string, The ISO 8061 UTC Timestamp of when the Mutation was started
- `UUID`: a string, A unique identifier for the Mutation
- `Original`: a binary blob, A commitment of the original mutable properties of the Branch Key
- `Terminal`: a binary blob, A commitment of the terminal mutable properties of the Branch Key
- `Input`: a binary blob, A Description of the input to initialize a Mutation
- `CiphertextBlob`: a binary blob, which MAY be used to protect the Mutation Commitment

### MutationIndex

A structure holding information on
an in-flight Mutation of a Branch Key
that is regularly changed in-flight.

The members are:

- `Identifier`: a string, The Branch Key (ID) under Mutation
- `CreateTime`: a string, The ISO 8061 UTC Timestamp of when the Mutation was started
- `UUID`: a string, A unique identifier for the Mutation
- `PageIndex`: a binary blob, Indirectly describes which items of the Branch Key have already been Mutated
- `CiphertextBlob`: a binary blob, which MAY be used to protect the Mutation Index

### OverWriteMutationIndex

A structure, much like
[OverWriteEncryptedHierarchicalKey](../../framework/key-store/key-storage.md#overwriteencryptedhierarchicalkey),
that affords a conditioned update to the Mutation Index.

The members are:

- `Index`: The new Mutation Index
- `Old`: The last read Mutation Index

### WriteInitializeMutationVersion

A structure that allows for a Mutation to either
create a new Version (DECRYPT*ONLY) of a Branch Key
or update the Active's Version (DECRYPT_ONLy) of a Branch Key,
via an \_Optimistic Lock*.

The members are:

- `rotate`: An [EncryptedHierarchicalKey](#encryptedhierarchicalkey) with a [type](#type) of HierarchicalSymmetricVersion
- `mutate`: An [OverWriteEncryptedHierarchicalKey](#overwriteencryptedhierarchicalkey) with both `Item` and `Old` with [type](#type) of HierarchicalSymmetricVersion

## Interface

The KeyStorageInterface MUST support the following operations:
Expand All @@ -84,6 +146,14 @@ The KeyStorageInterface MUST support the following operations:
- [GetEncryptedBranchKeyVersion](#getencryptedbranchkeyversion)
- [GetEncryptedBeaconKey](#getencryptedbeaconkey)
- [GetKeyStorageInfo](#getkeystorageinfo)
- [GetItemsForInitializeMutation](#getitemsforinitializemutation)
- [WriteInitializeMutation](#writeinitializemutation)
- [WriteAtomicMutation]
- [QueryForVersions]
- [WriteMutatedVersions]
- [GetMutation]
- [DeleteMutation]
- [WriteMutationIndex]

### WriteNewEncryptedBranchKey

Expand Down Expand Up @@ -124,3 +194,65 @@ It MUST return an [EncryptedHierarchicalKey](#encryptedhierarchicalkey).
### GetKeyStorageInfo

It MUST return the physical table name.

### GetItemsForInitializeMutation

The `GetItemsForInitializeMutation` caller MUST provide an `Identifer` that is the Branch Key ID.

The operation MUST fetch the following from the storage,
if they exist:

- [`MutationCommitment`](#mutationcommitment) for the `Identifier`
- [`MutationIndex`](#mutationindex) for the `Identifier`

The operation MUST fetch the following from the storage:

- The Active Branch Key for the `Identifier`
- The Active Beacon Key for the `Identifier`

If the Active Branch Key or Active Beacon Key are not found,
an error MUST be thrown.

If any of the items are malformed,
an error MUST be thrown.

Otherwise,
at least the Active Branch and Active Beacon Keys MUST be
returned as [EncryptedHierarchicalKey](#encryptedhierarchicalkey),
along with the Mutation Commitment or Mutation Index IF they are present.

### WriteInitializeMutation

The caller MUST provide:

- `ActiveItem`: An [OverWriteEncryptedHierarchicalKey](#overwriteencryptedhierarchicalkey)
- `BeaconItem`: An [OverWriteEncryptedHierarchicalKey](#overwriteencryptedhierarchicalkey)
- `Version`: An [WriteInitializeMutationVersion](#writeinitializemutationversion)
- `MutationCommitment`: A [MutationCommitment](#mutationcommitment)
- `MutationIndex`: A [MutationIndex](#mutationindex)

If any of these are missing or malformed,
an error MUST be thrown.

All of these structures MUST have the same Branch Key ID/Identifier.

The operation MUST atomically persist all 5 items
in one write.

The write MUST only succeed if:

- there is no existing Mutation Commitment for the Branch Key ID
- the CiphertextBlob of the Branch Key ID's Active is equal to `ActiveItem`'s `old` CipherText Blob.
- the CiphertextBlob of the Branch Key ID's Beacon is equal to `BeaconItem`'s `old` CipherText Blob.
- If the `Version` is `mutate`, the CiphertextBlob of the Version is equal to `mutate`'s `old` CipherText Blob.
- If the `Version` is `rotate`, there is no Branch Key Item with the same `type` value as `rotate`

If the write does not succeed because of the above constraints,
the operation MUST yield error;
this error SHOULD suggest a race is occurring.

If the write fails for other reasons,
the operation MUST yield error;
this error SHOULD suggest why the write failed.

Otherwise, the operation MUST return a success.

0 comments on commit 797a657

Please sign in to comment.