diff --git a/changes/2024-09-13_cache-across-hierarchical-keyrings/background.md b/changes/2024-09-13_cache-across-hierarchical-keyrings/background.md new file mode 100644 index 00000000..28fff359 --- /dev/null +++ b/changes/2024-09-13_cache-across-hierarchical-keyrings/background.md @@ -0,0 +1,419 @@ +[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved." +[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0" + +# Cache Across Hierarchical Keyrings + +## Background + +A [Cryptographic Materials Cache](https://github.com/awslabs/aws-encryption-sdk-specification/blob/master/framework/cryptographic-materials-cache.md) +can be used by a Hierarchical Keyring, a Caching CMM, and the DB-ESDK for Searchable Encryption. +It can store four types of [materials](https://github.com/aws/aws-cryptographic-material-providers-library/blob/c07a51fc29ff70411f7573bca96d2a091db8c1ed/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/Model/cryptographic-materials-cache.smithy#L89) +for three different use-cases: + +- Hierarchical Keyring: BranchKeyMaterials +- CachingCMM: EncryptionMaterials and DecryptionMaterials +- DB-ESDK (Searchable Encryption): BeaconKeyMaterials + +These materials have certain cache identifiers for accessing +them until their TTL expires. As of the date of creation of this document (09/13/12024), +the cache identifiers are set up such that: + +- Only one Hierarchical Keyring can use one cache, + i.e., currently, we do NOT support a shared cache + between multiple Hierarchical Keyrings. +- Multiple CachingCMMs can use a single shared cache + (this is for native language implementations only, + there is, currently, no CachingCMM implemented + in Dafny) +- We do NOT support a shared cache between multiple + Searchable Encryption Configurations. + +## Goals + +- We must allow caching across Key Stores/KMS Clients/KMS Keys + for multiple Hierarchical Keyrings and CachingCMMs. + For this, we need to strategically update the cache identifiers + for all materials ([background](#background)) in the CMC. +- We MUST prevent Confused Deputy cases, i.e: the wrong Branch Key + Material being served +- Our solution MUST be Easy to Use, Hard to Misuse + +## Out of Scope + +- Supporting a shared Cache in the DB-ESDK to fetch + BeaconKeyMaterials across multiple Searchable + Encryption Configurations, or across multiple KMS Relationships. + +## Issues and Alternatives: + +Before looking at the issues, please take a quick look at +[Hierarchical Keyring Cache Identifier formula](#hierarchical-keyring-cache-identifier-formula), +[CachingCMM Cache Identifier formula](#cachingcmm-cache-identifier-formula), and +[DB-ESDK Searchable Encryption Cache Identifier formula](#db-esdk-searchable-encryption-cache-identifier-formula) for more context. + +Note that there are currently two hashings taking place. +For example in the Hierarchical Keyring Cache Identifier formula, +one hash is for the branch-key-id and the other is hashing +the entire appended string +(a part of which is the hash of the branch-key-id) +to get the final cache identifier. +We'll call these internal and external hashes. + +_Preferred options are in italics._ + +### Issue 1: How to update the cache identifier for the Hierarchical Keyring to allow a shared cache? + +Note that according to the current +[Hierarchical Keyring Cache Identifier formula](#hierarchical-keyring-cache-identifier-formula), +if two KMS Relationships Hierarchical Keyrings are +called with two different Key Stores but the same branch-key-id, +there will be a collision (the branch-key-version will also +have to co-incide in case of Decryption Branch Key Materials). + +#### _Option 1 (Recommended): Adding a Resource ID, Scope ID, Partition ID, and Resource Suffix (recommended)_ + +We establish the following definitions for the +Cache Entry Identifier formula: + +- **Resource Identifier:** + A Hex value that indicates + if an element is from a Caching CMM, Hierarchical Keyring, + or some other future resource. + + - Caching_CMM : `0x01` (0001) + - Hierarchical_Keyring : `0x02` (0010) + +- **Scope Identifier:** + A Hex value that indicates + if an element is used for Encryption, Decryption, + Searchable Encryption, or some other future + purpose. + + - Encrypt : `0x01` (0001) + - Decrypt : `0x02` (0010) + - Searchable Encryption : `0x03` (0011) + +- **Partition ID:** + Either a v4 UUID, which SHOULD be interpreted + as the 16 byte representation of the UUID, or if + a String is provided by user, Partition ID MUST be + interpreted as the bytes of UTF-8 Encoding of the + String. + + Note: The Cache will not know if this ID is a + String set by the user or the UUID. + The constructor of the Hierarchical Keyring MUST + record these bytes at construction time. + This also makes the Hierarchical Keyring Partition ID + in-line with the Partition ID of the Caching CMM. + +- **Resource Suffix:** + There are, at this time, 5 resource suffixes: + - Hierarchical Keyring: Encryption Materials: + ``` + logicalKeyStoreName + NULL_BYTE + UTF8Encode(branchKeyId) + ``` + - Hierarchical Keyring: Decryption Materials: + ``` + logicalKeyStoreName + NULL_BYTE + UTF8Encode(branchKeyId) + NULL_BYTE + UTF8Encode(branchKeyVersion) + ``` + - Caching CMM: Encryption Materials, Without Algorithm Suite: + ``` + 0x00 + NULL_BYTE + SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext) + ``` + - Caching CMM: Encryption Materials, With Algorithm Suite: + ``` + 0x01 + NULL_BYTE + AlgorithmSuiteId(getEncryptionMaterialsRequest.algorithmSuite) + NULL_BYTE + SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext) + ``` + - Caching CMM: Decryption Materials: + ``` + AlgorithmSuiteId(decryptMaterialsRequest.algorithmSuite) + NULL_BYTE + CONCATENATE(SORTED(EDK)) + NULL_BYTE + SerializeEncryptionContext(decryptMaterialsRequest.encryptionContext) + ``` + +The final recommendation regarding cache identifiers +is to join the aforementioned 4 words +(Resource Identifier, Scope Identifier, Partition ID, +and Resource Suffix) with the null byte, 0x00, +and take the SHA384 of the result. + +#### Option 2: Using Key Store ID instead of Partition ID + +The Hierarchical Keyring holds the Key Store and the +underlying CMC. The Key Store ID is a parameter of +the keystore. Instead of saying that the +Hierarchical Keyring uses the Key Store ID from the +Key Store to determine if the cache is shared or not, +we should include a partition ID +(mentioned in the previous option), which is held directly +by the Hierarchical Keyring. + +This also, like mentioned before, makes the +Hierarchical Keyring cache identifier usage in-line with +the Caching CMM cache identifier. which uses the +partition ID for every caching CMM to determine +if caches are shared or not. + +Using the Key Store ID, +a property not necessarily bound to only one +instance of a Hierarchical Keyring, +introduces the following risks: + +- Time-To-Live is not a property of the Key Store, + but of the Hierarchical Keyring. Thus, two Hierarchical + Keyrings with different Time-To-Live configurations + but the same Key Store will share entries. This is + mitigated elsewhere, but it is good it is prevented + by the identifier as well. +- Branch Key ID Supplier is, likewise, not a property + of the Key Store but of the Hierarchical Keyring. + Thus, two Hierarchical Keyrings with the same Key Store + but different Branch Key ID Suppliers would share entries. + Though there is no threat, as the Branch Key ID Supplier + or statically bound Branch Key ID will prevent any out of + scope Branch Key IDs from being served. + +Ultimately, +it appears best to have the component that reads and +writes to the Cache identify itself, +as compared to using an Identity of a property/component +bound to it; +the Key Store only determines SOME of the properties of +the cached entries. +The Keyring determines ALL of the properties. + +#### Option 3: Double hashing the cache identifiers + +If you look at the cache identifiers right now, +they have a pattern of double hashing. +Double hashing, as discussed in the next issue, +is only done to prevent length extension attacks and +can be mitigated by a different (more cleaner) +technique: Using SHA384 instead of SHA512. + +### Issue 2: SHA512 vs SHA384 and do we need double hashing of cache identifiers? + +Before we discuss the differences between SHA512 +and SHA384, please note that SHA384 is +created by calculating a SHA512 hash and then truncating +to take the first 48 out of 64 bytes and ignoring +the other 16. + +If you look at the cache identifiers right now, +they have a pattern of double hashing. +For instance in the Hierarchical Keyring encrypt identifier, +the branch-key-digest (which is SHA(branch-key-id)) +is created and appended to the length of the branch +key and activeUtf8 (details of what is appended is +not important for this question and is discussed in Issue 1). +A hash of the appended blob is then taken to generate the +cache identifier. This means that the branch-key-digest is +hashed again after appending some other relevant bytes. + +#### _Option 1: SHA384 and no double hashing (recommended)_ + +SHA-384 is immune to length extension attacks and provides +enough cryptographic security for the hash. +Therefore, SHA384 is the preferred method. +Choosing SHA384 also helps in resolving [this issue](#how-can-we-ensure-that-there-are-no-cache-identifier-collisions-between-the-to-be-implemented-dafny-and-native-caching-cmms). + +#### Option 2: SHA512 + +SHA512 has a bigger output space [[ref](#birthday-bound-for-sha512-and-sha384)] +since the 64 byte output hash is not truncated to 48 bytes. +However, the additional output size doesn't add anything +and SHA384 uses the same algorithm. +The additional bytes can be removed to get immunity from +length extension attacks, among other things discussed above. + +### Issue 3: How to include the shared cache parameter in the Hierarchical Keyring input? + +#### _Option 1: Add it to the CacheType union in the [cryptographic-materials-cache.smithy](https://github.com/aws/aws-cryptographic-material-providers-library/blob/3ffe9f801fc625381d26aead2dc66e6e5cd83f1c/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/Model/cryptographic-materials-cache.smithy#L213) (recommended)_ + +This option allows us to have only one CacheType union. +This also gets rid of any confusions with using other types +of caches (like in the next option). + +#### Option 2: Add an optional input parameter to the CreateAwsKmsHierarchicalKeyringInput + +If we add an optional input parameter, customers will +see two optional input parameters for providing a cache, +first for providing a cache to be used by only one +Hierarchical Keyring, and second for providing a cache +potentially shared across Hierarchical Keyrings. +If the customer provides both, we will have to +throw an error. Option 1 buries this error one layer +deeper and is easier to use for the customer without +throwing unnecessary errors. With Option 1, we also avoid having two +input parameters that both take a cache, which isn't intuitive. + +### Issue 4: What should we name the shared cache parameter in the CacheType union? + +#### _Option 1: shared_ + +Shared cache aligns with what we want to say and it is +more intuitive. We are introducing a shared cache across +multiple Hierarchical Keyrings and should follow that +nomenclature. How the cache is provided are details that +we can mention in the examples / developer guide. + +#### Option 2: initialized + +For using the cache, the customer needs to initialize the +cache before the creation of the Hierarchical Keyring, when +in all the other cases the cache is initialized by the +Hierarchical Keyring and the customer just specifies the type. +One more consideration is that customers can create a +custom CMC and provide it to the initialized cache, +which does not need to be shared. + +## How can we ensure all the new cache identifiers are collision resistent? + +Based on [Option 1 in Issue 1](#option-1-recommended-adding-a-resource-id-scope-id--partition-id-and-resource-suffix-recommended), +the Resource ID and Scope ID make the +Caching CMM Encrypt / +Caching CMM Decrypt / +Hierarchical Keyring Encrypt +/ Hierarchical Keyring Decrypt distinct. + +## How can we ensure that there are no cache identifier collisions between the (to-be-implemented) Dafny and native Caching CMMs? + +The cache identifiers for the native Caching CMM +use SHA512 hash which has 64 bytes of output. +The new cache identifiers for the caching CMM use +SHA384 hash which has 48 bytes of output. +There will NEVER be a collision between the two. + +## Properties of Cryptographic Materials Cache + +There are three element of a cryptographic cache that +security engineers want to control: + +1. Time to Live (TTL) +2. Operation count +3. Total bytes through the system + +While we understand the importance of all these elements, +currently, only the TTL is in-scope for this change. +We MUST consider the other two when we look at the Caching CMM +because a lot of the implementation there will overlap with +that of the Hierarchical Keyring. + +## What if two Hierarchical Keyrings having different TTLs share a cache? + +TTL is provided as an optional parameter to a +Hierarchical Keyring at time of initialization. +As per the current implementation, if two Hierarchical Keyrings +have different TTLs, the cache will set the expiry time of +the cached material according to the TTL of the keyring that +populates the element in the cache. + +Therefore, if the other keyring with a different TTL +gets the element, the TTL will be wrong. + +Let us say that we have a Hierarchical Keyrings K1 and K2 with +TTLs 5000s and 5s. K1 populates the cache at time t = 0, +and K2 fetches materials from the cache at t = 10s. +Based on the current implementation, K2 will be able to +use a material which is older than the TTL specified in K2. + +We mitigate this by making sure that if the material exists +in the cache, the TTL has not expired for the keyring getting +the material. If it has, we basically assume that the material +is expired, and we re-populate the cache by decrypting +the BranchKeyMaterials again. + +## Cache Identifier Formulae + +### Hierarchical Keyring Cache Identifier formula + +- [Encrypt](https://github.com/aws/aws-cryptographic-material-providers-library/blob/c07a51fc29ff70411f7573bca96d2a091db8c1ed/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/src/Keyrings/AwsKms/AwsKmsHierarchicalKeyring.dfy#L384) + - [[spec]](https://github.com/awslabs/aws-encryption-sdk-specification/tree/ffb2b0cc6a956b2cec3a33be3c3672605b6907fb/framework/aws-kms/aws-kms-hierarchical-keyring.md#encryption-materials) + ``` + SHA512( + len_branch_key + + SHA512(branch_key_id_utf8) + + [0x00] + + activeUtf8 + )[0..32] + ``` + - The SHA512(branch_key_id_utf8) is called the branch-key-digest +- [Decrypt](https://github.com/aws/aws-cryptographic-material-providers-library/blob/c07a51fc29ff70411f7573bca96d2a091db8c1ed/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/src/Keyrings/AwsKms/AwsKmsHierarchicalKeyring.dfy#L725) + - [[spec]](https://github.com/awslabs/aws-encryption-sdk-specification/tree/ffb2b0cc6a956b2cec3a33be3c3672605b6907fb/framework/aws-kms/aws-kms-hierarchical-keyring.md#decryption-materials) + ``` + SHA512( + len_branch_key + + branch_key_id_utf8 + + [0x00 as uint8] + + UTF8.EncodeAscii(branchKeyVersion) + )[0..32] + ``` +- The length of cache identifier for BranchKeyMaterials + is 32 bytes for both Encrypt and Decrypt. + +### CachingCMM Cache Identifier formula + +- For python native (Note: There is no CachingCMM implemented in dafny yet): + - [Encrypt](https://github.com/aws/aws-encryption-sdk-python/blob/1a1213a31776477dcc4aab44b2a4dc2eb514113e/src/aws_encryption_sdk/caches/__init__.py#L55): as per [spec](https://github.com/awslabs/aws-encryption-sdk-specification/tree/ffb2b0cc6a956b2cec3a33be3c3672605b6907fb/framework/caching-cmm.md#appendix-a-cache-entry-identifier-formulas) + - Encrypt without Algorithm suite specified + ``` + SHA512( + SHA512(UTF8Encode(cachingCMM.partitionId)) + + 0x00 + + SHA512(SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext)) + ) + ``` + - Encrypt with Algorithm suite specified + ``` + SHA512( + SHA512(UTF8Encode(cachingCMM.partitionId)) + + 0x01 + + AlgorithmSuiteId(getEncryptionMaterialsRequest.algorithmSuite) + + SHA512(SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext)) + ) + ``` + - [Decrypt](https://github.com/aws/aws-encryption-sdk-python/blob/1a1213a31776477dcc4aab44b2a4dc2eb514113e/src/aws_encryption_sdk/caches/__init__.py#L101): as per [spec](https://github.com/awslabs/aws-encryption-sdk-specification/tree/ffb2b0cc6a956b2cec3a33be3c3672605b6907fb/framework/caching-cmm.md#appendix-a-cache-entry-identifier-formulas) + ``` + EDK_HASHES = [SHA512(SerializeEncryptedDataKey(key)) for key in decryptMaterialsRequest.encryptedDataKeys] + ENTRY_ID = SHA512( + SHA512(UTF8Encode(cachingCMM.partitionId)) + + AlgorithmSuiteId(decryptMaterialsRequest.algorithmSuite) + + CONCATENATE(SORTED(EDK_HASHES)) + + PADDING_OF_512_ZERO_BITS + + SHA512(SerializeEncryptionContext(decryptMaterialsRequest.encryptionContext)) + ) + ``` +- The length of cache identifier for EncryptionMaterials + and DecryptionMaterials is 64 bytes. + +### DB-ESDK Searchable Encryption Cache Identifier formula + +- To fetch BeaconKeyMaterials from the + CMC for Searchable Encryption, + the cache identifier for a beacon key is a `key-id`. The key-id is a + [string](https://github.com/aws/aws-database-encryption-sdk-dynamodb/blob/b5705ee12257fb18f867478bf17ba31f50c26c8b/DynamoDbEncryption/dafny/DynamoDbEncryption/src/SearchInfo.dfy#L185), + and here is an example initialization + [[ref](https://github.com/aws/aws-database-encryption-sdk-dynamodb/blob/b5705ee12257fb18f867478bf17ba31f50c26c8b/TestVectors/dafny/DDBEncryption/src/JsonConfig.dfy#L527)]. +- The length of cache identifier for BeaconKeyMaterials is “variable” based on the key-id. +- For single-tenant, the customer sets the key-id directly in the config. + or multi-tenant, they set an attribute name, and the key comes out of that attribute of the item. + +## Birthday Bound for SHA512 and SHA384 + +A [birthday attack](https://en.wikipedia.org/wiki/Birthday_attack) +is a bruteforce collision attack that exploits the +mathematics behind the birthday problem in probability +theory. This attack can be used to abuse communication +between two or more parties. The attack depends on the +higher likelihood of collisions found between random +attack attempts and a fixed degree of +permutations (pigeonholes). With a birthday attack for a hash function, +if we have total n bits of output and therefore +$2^n$ different outputs, it is possible to find a +collision of the hash function with 50% chance by +calculating approximately $sqrt(2^n) = 2^{n/2}$ hashes. + +To find a random collision in a SHA512 hash with +probability 0.5, we would need to calculate +approximately $2^256$ hashes. Similarly for SHA384, +we would need to calculate $2^192$ hashes. +Therefore, SHA512 is more collision resistant. diff --git a/changes/2024-09-13_cache-across-hierarchical-keyrings/change.md b/changes/2024-09-13_cache-across-hierarchical-keyrings/change.md new file mode 100644 index 00000000..8504ae51 --- /dev/null +++ b/changes/2024-09-13_cache-across-hierarchical-keyrings/change.md @@ -0,0 +1,216 @@ +[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved." +[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0" + +# Cache Across Hierarchical Keyrings + +## Affected Features + +This serves as a reference of all features that this change affects. + +| Feature | +| --------------------------------------------------------------------------------------- | +| [AWS KMS Hierarchical Keyring](../../framework/aws-kms/aws-kms-hierarchical-keyring.md) | +| [Caching Cryptographic Materials Manager](../../framework/caching-cmm.md) | + +## Affected Specifications + +This serves as a reference of all specification documents that this change affects. + +| Specification | +| --------------------------------------------------------------------------------------- | +| [AWS KMS Hierarchical Keyring](../../framework/aws-kms/aws-kms-hierarchical-keyring.md) | +| [Caching Cryptographic Materials Manager](../../framework/caching-cmm.md) | + +## Affected Implementations + +| Language | Version Introduced | Version Removed | Implementation | +| -------- | ------------------ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Dafny | 1.7.0 | n/a | [AwsKmsHierarchicalKeyring.dfy](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/src/Keyrings/AwsKms/AwsKmsHierarchicalKeyring.dfy) | +| Java | 1.7.0 | n/a | [CreateAwsKmsHierarchicalKeyringInput.java](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/materialproviders/model/CreateAwsKmsHierarchicalKeyringInput.java) | +| .NET | 1.7.0 | n/a | [CreateAwsKmsHierarchicalKeyringInput.cs](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/net/Generated/AwsCryptographicMaterialProviders/CreateAwsKmsHierarchicalKeyringInput.cs) | + +## Definitions + +An "MPL Consumer" is a library, service, or other application +that uses the AWS Cryptographic Material Providers Library (MPL) +to manage cryptographic materials. +An "MPL Consumer" MAY be using an AWS Crypto Tools product, +such as the AWS Encryption SDK or AWS Database Encryption SDK. + +By "KMS Relationship", we mean any or all of the following: + +- KMS Configuration +- Credentials used when creating the KMS Client, and thus used when calling KMS +- Other properties of the KMS Client, such as the region, or request headers + +### Conventions used in this document + +The key words +"MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", +"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" +in this document are to be interpreted as described in +[RFC 2119](https://tools.ietf.org/html/rfc2119). + +## Summary + +To improve the MPL Consumer data key caching experience, +when using the Hierarchical Keyring, +we need to allow caching across Key Stores/KMS Clients/KMS Keys. + +To facilitate caching across Key Stores/KMS Clients/KMS Keys, +we MUST break the Cryptographic Materials Cache (CMC) +out of the Hierarchical Keyring. + +By allowing MPL Consumers to optionally provide an initialized shared CMC +to the Hierarchical Keyring at construction, +the CMC MAY cache Branch Keys protected by different +KMS Relationships. + +This simplifies Multiple KMS Relationship MPL Consumers, +as they do not need to stand up an LRU Cache of Hierarchical Keyrings. + +Instead, they may maintain one CMC. +They still create a Hierarchical Keyring instance per KMS Relationship, +and they MUST use the correct Keyring to retrieve material +from the Cache. + +But they only need to maintain the common cache. + +This change also means that any keyring that has access to the `Shared` cache +MAY be able to use materials that it MAY or MAY NOT have direct access to. +Therefore, it is important to carefully configure the Partition ID, +Logical Key Store Name of the Key Store for the Hierarchical Keyring, and the +Branch Key ID for the Hierarchical Keyrings sharing a cache. +Please review the [Security Implications](#security-implications) for more +information on this. + +In the future, the CachingCMM will be introduced to +Crypto Tool's Dafny products. +The CachingCMM and the Hierarchical Keyring both consume CMCs. +Thus, it will be possible to provide a CMC to both a +Hierarchical Keyring & a CachingCMM. +This cache identifier design considers +how the cache entries of multiple Hierarchical Keyrings +and CachingCMMs will be appropriately separated. + +## Out of Scope + +- Supporting a shared Cache in the DB-ESDK to fetch + BeaconKeyMaterials across multiple Searchable + Encryption Configurations, or across multiple KMS Relationships. + +## Motivation + +The Hierarchical Keyring, +and it's component the Key Store, +allow MPL Consumers to reduce their KMS Call volume +by persisting KMS protected cryptographic materials into +an available medium +(currently, only a DynamoDB table is available as persistence medium). + +We call these cryptographic materials Branch Keys. + +However, an instance of the Hierarchical Keyring +can only ever call KMS with one KMS Relationship, +which is, at least partly, +configured on the KMS Client determined +at the Hierarchical Keyring's construction. + +By KMS Relationship, we mean any or all of the following: + +- KMS Configuration +- Credentials used when creating the KMS Client, and thus + used when calling KMS +- Other properties of the KMS Client, such as the region, + or request headers + +The Local Cryptographic Material Cache of +the Hierarchical Keyring instance is then only +populated with Branch Keys that correspond with +that KMS relationship. + +Which is appropriate, +as it is clear under what KMS relationship +a Branch Key is accessed. + +However, +the Hierarchical Keyring, +and it's Key Store, +have a runtime cost, +exerting memory pressure +and, without manual optimization, +requiring at least 2 TLS handshakes +when first serving a request +(TLS to KMS & TLS to DDB). + +Additionally, +the local Cryptographic Materials Cache +exerts some runtime cost, +particularly in a multi-threaded environment, +when a background worker thread MAY be refreshing +or pruning entries of the cache. + +For MPL Consumers that MUST work with Branch Keys +under different KMS Relationships, +this runtime cost adds up. + +These MPL Consumers MAY end up establishing +an LRU Cache of Hierarchical Keyrings. +Which, while workable, is sub-optimal, +and clearly makes the Hierarchical Keyring, +in these conditions, +"Hard to Use". + +The objective, with these changes, +is to make the Hierarchical Keyring +"Easy to Use" in a multiple KMS Relationship +environment. + +## Security Implications + +The main security implication of this change is the responsibility for +providing the Partition ID for the Hierarchical Keyring, +and the Logical Key Store Name of the Key Store for the +Hierarchical Keyring while using a `Shared` cache. + +Users need to be careful while setting the Partition ID and +Logical Key Store Name because if there are two or more Hierarchical Keyrings with: + +- Same Partition ID +- Same Logical Key Store Name of the Key Store for the Hierarchical Keyring +- Same Branch Key ID + +then the Hierarchical Keyrings WILL share cache entries in the `Shared` Cache. + +This means that the branch-key used and cached by one Hierarchical Keyring can be +used by the other Hierarchical Keyring (within a TTL), effectively by-passing +KMS Access Control. We recommend evaluating your threat model carefully, to +understand and mitigate this risk yourself. By default, each Hierarchical Keyring's +Partition ID is set to a v4 UUID, which is a random 16 byte representation of the UUID. +Unless the customer explicitly sets the Partition ID of two Hierarchical Keyrings to +be the same, Hierarchical Keyrings will NOT by-pass KMS Access Controls. + +Following is another important security consideration. Suppose you create two Hierarchical Keyrings +with the same Partition ID. The two keyrings are backed by two different physical DynamoDB Key Stores +holding two different branch keys, but with the same Branch Key ID for both the Branch Keys. +Additionally, the two different physical DynamoDB Key Stores also have the same +Logical Key Store Name. In this case, there is a risk of accidental unwanted cache +collisions where truly distinct cryptographic materials can be stored under the +same Cache Identifier. This is a security risk and can lead to customers encrypting their data +with the wrong branch key, which they MAY NOT have access to. +This risk can be mitigated by following CryptoTool's guidance: +There MUST always be a one-one mapping of your DynamoDB Key Store and the Logical Key Store Name. +That is, you should NEVER have two DynamoDB Key Stores with the same Logical Key Store Name. + +Users should make sure that they set all of Partition ID, Logical Key Store Name and Branch Key ID +to be the same for two Hierarchical Keyrings if and only if they want them to share cache entries. + +## Operational Implications + +This change will allow customers to share an already initialized cache across multiple +Hierarchical Keyrings to facilitate caching across Key Stores/KMS Clients/KMS Keys. + +## Examples + +As part of this change, we will add two examples, one each in ESDK .NET and ESDK Java +demonstrating the use of `Shared` cache across Hierarchical Keyrings. diff --git a/framework/aws-kms/aws-kms-hierarchical-keyring.md b/framework/aws-kms/aws-kms-hierarchical-keyring.md index 578d0853..b1418964 100644 --- a/framework/aws-kms/aws-kms-hierarchical-keyring.md +++ b/framework/aws-kms/aws-kms-hierarchical-keyring.md @@ -5,17 +5,23 @@ ## Version -0.1.0 +0.2.0 ### Changelog +- 0.2.0 + - [Update Cache Entry Identifier Formulas to shared cache across multiple Hierarchical Keyrings](../../changes/2024-09-13_cache-across-hierarchical-keyrings/change.md) + - New optional parameter `Partition ID` used to distinguish Cryptographic Material Providers (i.e: Hierarchical Keyrings) writing to a cache - 0.1.0 - Initital record ## Implementations -| Language | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation | -| -------- | -------------------------------------- | ------------------------- | -------------- | +| Language | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation | +| -------- | -------------------------------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Dafny | 0.2.0 | 1.0.0 | [AwsKmsHierarchicalKeyring.dfy](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/src/Keyrings/AwsKms/AwsKmsHierarchicalKeyring.dfy) | +| Java | 0.1.0 | 1.0.0 | [CreateAwsKmsHierarchicalKeyringInput.java](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/materialproviders/model/CreateAwsKmsHierarchicalKeyringInput.java) | +| .NET | 0.1.0 | 1.0.0 | [CreateAwsKmsHierarchicalKeyringInput.cs](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/net/Generated/AwsCryptographicMaterialProviders/CreateAwsKmsHierarchicalKeyringInput.cs) | ## Overview @@ -47,7 +53,8 @@ On initialization, the caller: - MUST provide a [Keystore](../branch-key-store.md) - MUST provide a [cache limit TTL](#cache-limit-ttl) - MUST provide either a Branch Key Identifier or a [Branch Key Supplier](#branch-key-supplier) -- MAY provide a max cache size +- MAY provide a [Cache Type](#cache-type) +- MAY provide a [Partition ID](#partition-id) If the cache to initialize is a [Storm Tracking Cryptographic Materials Cache](../storm-tracking-cryptographic-materials-cache.md#overview) then the settings on the storm tracking cache need to be rational with respect to the settings for the keyring. @@ -61,12 +68,39 @@ On initialization the Hierarchical Keyring MUST initialize a [cryptographic-mate If no max cache size is provided, the crypotgraphic materials cache MUST be configured to a max cache size of 1000. +If the Hierarchical Keyring does NOT get a `Shared` cache on initialization, +it MUST initialize a [cryptographic-materials-cache](../local-cryptographic-materials-cache.md) +with the user provided cache limit TTL and the entry capacity. +If no `cache` is provided, a `DefaultCache` MUST be configured with entry capacity of 1000. + ### Cache Limit TTL The maximum amount of time in seconds that an entry within the cache may be used before it MUST be evicted. The client MUST set a time-to-live (TTL) for [branch key materials](../structures.md#branch-key-materials) in the underlying cache. This value MUST be greater than zero. +### Cache Type + +Sets the type of cache for this Hierarchical Keyring. By providing an already initialized `Shared` cache, +users can determine the scope of the cache. That is, if the cache is shared across other Cryptographic Material Providers, +for instance other Hierarchical Keyrings or Caching Cryptographic Materials Managers (Caching CMMs). +If any other type of cache in the `CacheType` union is provided, the Hierarchical Keyring will initialize a +cache of that type, to be used with only this Hierarchical Keyring. If not set, a `DefaultCache` is initialized +to be used with only this Hierarchical Keyring with `entryCapacity` = 1000. + +### Partition ID + +An optional string that uniquely identifies the respective Hierarchical Keyring +and is used to avoid collisions with other Hierarchical Keyrings. + +PartitionId can be a string provided by the user. If provided, it MUST be interpreted as UTF8 bytes. +If the PartitionId is NOT provided by the user, it MUST be set to the 16 byte representation of a v4 UUID. + +The Partition ID MUST NOT be changed after initialization. + +Please see [Shared Cache Considerations](#shared-cache-considerations) on how to provide the +Partition ID and Logical Key Store Name while providing a Shared Cache to the Hierarchical Keyring. + ## Structure ### Ciphertext @@ -122,6 +156,14 @@ to compute the [cache entry identifier](../cryptographic-materials-cache.md#cach If a cache entry is found and the entry's TTL has not expired, the hierarchical keyring MUST use those branch key materials for key wrapping. +If using a `Shared` cache across multiple Hierarchical Keyrings, +different keyrings having the same `branchKey` can have different TTLs. +In such a case, the expiry time in the cache is set according to the Keyring that populated the cache. +There MUST be a check (cacheEntryWithinLimits) to make sure that for the cache entry found, who's TTL has NOT expired, +`time.now() - cacheEntryCreationTime <= ttlSeconds` is true and +valid for TTL of the Hierarchical Keyring getting the cache entry. +If this is NOT true, then we MUST treat the cache entry as expired. + If a cache entry is not found or the cache entry is expired, the hierarchical keyring MUST attempt to obtain the branch key materials by querying the backing branch keystore specified in the [retrieve OnEncrypt branch key materials](#query-branch-keystore-onencrypt) section. @@ -206,6 +248,14 @@ in order to compute the [cache entry identifier](cryptographic-materials-cache.m If a cache entry is found and the entry's TTL has not expired, the hierarchical keyring MUST use those branch key materials for key unwrapping. +If using a `Shared` cache across multiple Hierarchical Keyrings, +different keyrings having the same `branchKey` can have different TTLs. +In such a case, the expiry time in the cache is set according to the Keyring that populated the cache. +There MUST be a check (cacheEntryWithinLimits) to make sure that for the cache entry found, who's TTL has NOT expired, +`time.now() - cacheEntryCreationTime <= ttlSeconds` is true and +valid for TTL of the Hierarchical Keyring getting the cache entry. +If this is NOT true, then we MUST treat the cache entry as expired. + If a cache entry is not found or the cache entry is expired, the hierarchical keyring MUST attempt to obtain the branch key materials by calling the backing branch key store specified in the [retrieve OnDecrypt branch key materials](#getitem-branch-keystore-ondecrypt) section. @@ -302,66 +352,213 @@ in order to compute the [cache entry identifier](../cryptographic-materials-cach Each of the cache entry identifier formulas includes serialized information related to the branch key, as defined in the [Key Provider Info](../structures.md#key-provider-information). -### Encryption Materials +We establish the following definitions for the Cache Entry Identifier formula: -When the hierarchical keyring receives an OnEncrypt request, -the cache entry identifier MUST be calculated as the first 32 bytes of the -SHA-512 hash of the following byte strings, in the order listed: +#### Resource Identifier + +A Hex value that indicates if an element is from a Caching_CMM, Hierarchical_Keyring, or some other future resource. + +``` +Caching_CMM : 0x01 (0001) +Hierarchical_Keyring : 0x02 (0010) +``` + +#### Scope Identifier + +A Hex value that indicates if an element is used for Encryption, Decryption, Searchable Encryption, or some other future purpose. + +``` +Encrypt : 0x01 (0001) +Decrypt : 0x02 (0010) +Searchable Encryption : 0x03 (0011) +``` + +#### Partition ID + +Partition ID is an optional parameter provided to the Hierarchical Keyring input, which distinguishes +Cryptographic Material Providers (i.e: Hierarchical Keyrings) writing to a cache. +It can either be a String provided by the user, which MUST be interpreted as the bytes of +UTF-8 Encoding of the String, or a v4 UUID, which SHOULD be interpreted as the 16 byte representation of the UUID. + +Note: The cache will not know if the Partition ID is a String set by the user or the UUID. +The constructor of the Hierarchical Keyring MUST record these bytes at construction time. + +Please see [Shared Cache Considerations](#shared-cache-considerations) on how to provide the +Partition ID and Logical Key Store Name while providing a Shared Cache to the Hierarchical Keyring. -- MUST be the length of the branch key id in its uint8 interpretation -- MUST be the SHA512 output of the UTF8 encoded branch-key-id -- MUST be a single null byte `0x00` -- MUST be the constant UTF8 encoded string "ACTIVE" +#### Resource Suffix -| Field | Length (bytes) | Interpreted as | -| ------------------------ | -------------- | -------------- | -| Length of branch-key-id | 3 | UInt8 | -| SHA-512(branch-key-id) | 64 | bytes | -| Null Byte | 1 | `0x00` | -| Constant string "ACTIVE" | 6 | UTF-8 Encoded | +There are two resource suffixes for the Hierarchical Keyring: + +- Hierarchical Keyring: Encryption Materials: + ``` + logicalKeyStoreName + NULL_BYTE + UTF8Encode(branchKeyId) + ``` +- Hierarchical Keyring: Decryption Materials: + ``` + logicalKeyStoreName + NULL_BYTE + UTF8Encode(branchKeyId) + NULL_BYTE + UTF8Encode(branchKeyVersion) + ``` + +The aforementioned 4 definitions ([Resource Identifier](#resource-identifier), +[Scope Identifier](#scope-identifier), [Partition ID](#partition-id-1), and +[Resource Suffix](#resource-suffix)) MUST be appended together with the null byte, 0x00, +and the SHA384 of the result should be taken as the final cache identifier. + +### Encryption Materials + +When the hierarchical keyring receives an OnEncrypt request, +the cache entry identifier MUST be calculated as the +SHA-384 hash of the following byte strings, in the order listed: + +- MUST be the Resource ID for the Hierarchical Keyring (0x02) +- MUST be the Scope ID for Encrypt (0x01) +- MUST be the Partition ID for the Hierarchical Keyring +- Resource Suffix + - MUST be the UTF8 encoded Logical Key Store Name of the keystore for the Hierarchical Keyring + - MUST be the UTF8 encoded branch-key-id + +All the above fields must be separated by a single NULL_BYTE `0x00`. + +| Field | Length (bytes) | Interpreted as | +| ---------------------- | -------------- | ------------------- | +| Resource ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Scope ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Partition ID | Variable | bytes | +| Null Byte | 1 | `0x00` | +| Logical Key Store Name | Variable | UTF-8 Encoded Bytes | +| Null Byte | 1 | `0x00` | +| Branch Key ID | Variable | UTF-8 Encoded Bytes | As a formula: ``` +resource-id = [0x02] +scope-id = [0x01] +logical-key-store-name = UTF8Encode(keystore.LogicalKeyStoreName) branch-key-id = UTF8Encode(hierarchicalKeyring.BranchKeyIdentifier) -branch-key-digest = SHA512(branch-key-id) - -ENTRY_ID = SHA512( - LengthUint8(branch-key-id) + - branch-key-digest + - + 0x00 - + UTF8Encode("ACTIVE") -)[0:32] +NULL_BYTE = [0x00] + +ENTRY_ID = SHA384( + resource-id + + NULL_BYTE + + scope-id + + NULL_BYTE + + partition-id + + NULL_BYTE + + logical-key-store-name + + NULL_BYTE + + branch-key-id +) ``` ### Decryption Materials When the hierarchical keyring receives an OnDecrypt request, -it MUST calculate the cache entry identifier as the first 32 bytes of the SHA-512 hash of the following byte strings, in the order listed: - -- MUST be the length of the branch key id in its uint8 interpretation -- MUST be the UTF8 encoded branch-key-id -- MUST be a single null byte `0x00` -- MUST be the UTF8 encoded branch-key-version +it MUST calculate the cache entry identifier as the +SHA-384 hash of the following byte strings, in the order listed: + +- MUST be the Resource ID for the Hierarchical Keyring (0x02) +- MUST be the Scope ID for Decrypt (0x02) +- MUST be the Partition ID for the Hierarchical Keyring +- Resource Suffix + - MUST be the UTF8 encoded Logical Key Store Name of the keystore for the Hierarchical Keyring + - MUST be the UTF8 encoded branch-key-id + - MUST be the UTF8 encoded branch-key-version + +All the above fields must be separated by a single NULL_BYTE `0x00`. + +| Field | Length (bytes) | Interpreted as | +| ---------------------- | -------------- | ------------------- | +| Resource ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Scope ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Partition ID | Variable | bytes | +| Null Byte | 1 | `0x00` | +| Logical Key Store Name | Variable | UTF-8 Encoded Bytes | +| Null Byte | 1 | `0x00` | +| Branch Key ID | Variable | UTF-8 Encoded Bytes | +| Null Byte | 1 | `0x00` | +| branch-key-version | 36 | UTF-8 Encoded Bytes | -| Field | Length (bytes) | Interpreted as | -| ----------------------- | -------------- | -------------- | -| Length of branch-key-id | 3 | UInt8 | -| branch-key-id | Variable | UTF-8 Encoded | -| Null Byte | 1 | `0x00` | -| branch-key-version | 36 | UTF-8 Encoded | +As a formula: ``` -branch-key-id = UTF8Encode(edk.providerInfo) - -ENTRY_ID = SHA512( - LengthUint8(branch-key-id) + - branch-key-id + - 0x00 + - UTF8Encode(branch-key-version) -)[0:32] +resource-id = [0x02] +scope-id = [0x02] +logical-key-store-name = UTF8Encode(keystore.LogicalKeyStoreName) +branch-key-id = UTF8Encode(hierarchicalKeyring.BranchKeyIdentifier) +branch-key-version = UTF8Encode(branchKeyVersion) +NULL_BYTE = [0x00] + +ENTRY_ID = SHA384( + resource-id + + NULL_BYTE + + scope-id + + NULL_BYTE + + partition-id + + NULL_BYTE + + logical-key-store-name + + NULL_BYTE + + branch-key-id + + NULL_BYTE + + branch-key-version +) ``` +## Shared Cache Considerations + +If a user has two or more Hierarchical Keyrings with: + +- Same Partition ID +- Same Logical Key Store Name of the Key Store for the Hierarchical Keyring +- Same Branch Key ID + +then they WILL share the cache entries in the `Shared` Cache. + +Any keyring that has access to the `Shared` cache MAY be able to use materials +that it MAY or MAY NOT have direct access to. + +Users MUST make sure that all of Partition ID, Logical Key Store Name of the Key Store for the Hierarchical Keyring +and Branch Key ID are set to be the same for two Hierarchical Keyrings if and only they want the keyrings to share +cache entries. + +Therefore, there are two important parameters that users need to carefully set while providing the shared cache: + +### Partition ID + +Partition ID is an optional parameter provided to the Hierarchical Keyring input, +which distinguishes Cryptographic Material Providers (i.e: Hierarchical Keyrings) writing to a cache. + +- (Default) A a random 16-byte UUID, which makes + it unique for every Hierarchical Keyring. In this case, two Hierarchical Keyrings (or another Material Provider) + CANNOT share the same cache entries in the cache. +- If the Partition ID is set by the user and is the same for two Hierarchical Keyrings (or another Material Provider), + they CAN share the same cache entries in the cache. +- If the Partition ID is set by the user and is different for two Hierarchical Keyrings (or another Material Provider), + they CANNOT share the same cache entries in the cache. + +### Logical Key Store Name + +> Note: Users MUST NEVER have two different physical Key Stores with the same Logical Key Store Name. + +Logical Key Store Name is set by the user when configuring the Key Store for +the Hierarchical Keyring. This is a logical name for the key store. +Logical Key Store Name MUST be converted to UTF8 Bytes to be used in +the cache identifiers. + +Suppose there's a physical Key Store on DynamoDB (K). Two Key Store clients of K (K1 and K2) are created. +Now, two Hierarchical Keyrings (HK1 and HK2) are created with these Key Store clients (K1 and K2 respectively). + +- If we want to share cache entries across these two keyrings HK1 and HK2, the Logical Key Store Names + for both the Key Store clients (K1 and K2) should be set to be the same. +- If we set the Logical Key Store Names for K1 and K2 to be different, HK1 (which uses Key Store client K1) + and HK2 (which uses Key Store client K2) will NOT be able to share cache entries. + +Notice that both K1 and K2 are clients for the same physical Key Store (K). + ## Branch Key Supplier The Branch Key Supplier is an interface containing the `GetBranchKeyId` operation. diff --git a/framework/caching-cmm.md b/framework/caching-cmm.md index 10ec2c67..bf5a356c 100644 --- a/framework/caching-cmm.md +++ b/framework/caching-cmm.md @@ -5,10 +5,12 @@ ## Version -0.3.0 +0.4.0 ### Changelog +- 0.4.0 + - [Update Cache Entry Identifier Formulas for Caching Cryptographic Materials Manager](../changes/2024-09-13_cache-across-hierarchical-keyrings/change.md) - 0.3.0 - [Specify Cache Entry Identifier Formulas for Caching Cryptographic Materials Manager](../changes/2020-07-17_cache-entry-identifier-formulas/change.md) - 0.2.0 @@ -197,24 +199,101 @@ to the denote the function that, given an algorithm suite as specified in a materials request, returns the corresponding two-byte algorithm suite ID. +We establish the following definitions for the Cache Entry Identifier formula: + +#### Resource Identifier + +A Hex value that indicates if an element is from a Caching_CMM, Hierarchical_Keyring, or some other future resource. + +``` +Caching_CMM : 0x01 (0001) +Hierarchical_Keyring : 0x02 (0010) +``` + +#### Scope Identifier + +A Hex value that indicates if an element is used for Encryption, Decryption, Searchable Encryption, or some other future purpose. + +``` +Encrypt : 0x01 (0001) +Decrypt : 0x02 (0010) +Searchable Encryption : 0x03 (0011) +``` + +#### Partition ID + +Partition ID of the Caching CMM. + +#### Resource Suffix + +There are three resource suffixes for the Caching CMM: + +- Caching CMM: Encryption Materials, Without Algorithm Suite: + ``` + 0x00 + NULL_BYTE + SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext) + ``` +- Caching CMM: Encryption Materials, With Algorithm Suite: + ``` + 0x01 + NULL_BYTE + AlgorithmSuiteId(getEncryptionMaterialsRequest.algorithmSuite) + NULL_BYTE + SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext) + ``` +- Caching CMM: Decryption Materials: + ``` + AlgorithmSuiteId(decryptMaterialsRequest.algorithmSuite) + NULL_BYTE + CONCATENATE(SORTED(EDK)) + NULL_BYTE + SerializeEncryptionContext(decryptMaterialsRequest.encryptionContext) + ``` + +The aforementioned 4 definitions ([Resource Identifier](#resource-identifier), +[Scope Identifier](#scope-identifier), [Partition ID](#partition-id-1), +and [Resource Suffix](#resource-suffix)) MUST be appended together with the null byte, +0x00, and the SHA384 of the result should be taken as the final cache identifier. + ### Encryption Materials, Without Algorithm Suite If the Get Encryption Materials request does not specify an algorithm suite, then the cache entry identifier MUST be calculated -as the SHA-512 hash of the concatenation of the following byte strings, +as the SHA-384 hash of the concatenation of the following byte strings, in the order listed: -1. The SHA-512 hash of a UTF-8 encoding of the caching CMM’s Partition ID -2. One null byte (`0x00`) -3. The SHA-512 hash of the serialized encryption context +- MUST be the Resource ID for the Caching CMM (0x01) +- MUST be the Scope ID for Encrypt (0x01) +- MUST be the UTF-8 encoding of the caching CMM’s Partition ID +- Resource Suffix + - MUST be the Algorithm Suite Hex byte `0x00` to indicate Encryption Materials Without Algorithm Suite + - MUST be the serialized encryption context + +All the above fields must be separated by a single NULL_BYTE `0x00`. + +| Field | Length (bytes) | Interpreted as | +| -------------------------- | -------------- | -------------- | +| Resource ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Scope ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Partition ID | Variable | bytes | +| Null Byte | 1 | `0x00` | +| Algorithm Suite Hex byte | 1 | bytes (`0x00`) | +| Null Byte | 1 | `0x00` | +| SerializeEncryptionContext | Variable | bytes | As a formula: ``` -ENTRY_ID = SHA512( - SHA512(UTF8Encode(cachingCMM.partitionId)) - + 0x00 - + SHA512(SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext)) +resource-id = [0x01] +scope-id = [0x01] +partition-id = UTF8Encode(cachingCMM.partitionId) +algorithm-suite-hex-byte = [0x00] +serialized-encryption-context = SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext) +NULL_BYTE = [0x00] + +ENTRY_ID = SHA384( + resource-id + + NULL_BYTE + + scope-id + + NULL_BYTE + + partition-id + + NULL_BYTE + + algorithm-suite-hex-byte + + NULL_BYTE + + serialized-encryption-context ) ``` @@ -222,22 +301,56 @@ ENTRY_ID = SHA512( If the Get Encryption Materials request does specify an algorithm suite, then the cache entry identifier MUST be calculated -as the SHA-512 hash of the concatenation of the following byte strings, +as the SHA-384 hash of the concatenation of the following byte strings, in the order listed: -1. The SHA-512 hash of a UTF-8 encoding of the caching CMM’s Partition ID -2. One byte with value 1 (`0x01`) -3. The two-byte algorithm suite ID corresponding to the algorithm suite in the request -4. The SHA-512 hash of the serialized encryption context +- MUST be the Resource ID for the Caching CMM (0x01) +- MUST be the Scope ID for Encrypt (0x01) +- MUST be the UTF-8 encoding of the caching CMM’s Partition ID +- Resource Suffix + - MUST be the Algorithm Suite Hex byte `0x01` to indicate Encryption Materials With Algorithm Suite + - MUST be the two-byte Algorithm Suite ID corresponding to the algorithm suite in the request + - MUST be the serialized encryption context + +All the above fields must be separated by a single NULL_BYTE `0x00`. + +| Field | Length (bytes) | Interpreted as | +| -------------------------- | -------------- | -------------- | +| Resource ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Scope ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Partition ID | Variable | bytes | +| Null Byte | 1 | `0x00` | +| Algorithm Suite Hex byte | 1 | bytes (`0x01`) | +| Null Byte | 1 | `0x00` | +| Algorithm Suite ID | 2 | bytes | +| Null Byte | 1 | `0x00` | +| SerializeEncryptionContext | Variable | bytes | As a formula: ``` -ENTRY_ID = SHA512( - SHA512(UTF8Encode(cachingCMM.partitionId)) - + 0x01 - + AlgorithmSuiteId(getEncryptionMaterialsRequest.algorithmSuite) - + SHA512(SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext)) +resource-id = [0x01] +scope-id = [0x01] +partition-id = UTF8Encode(cachingCMM.partitionId) +algorithm-suite-hex-byte = [0x01] +algorithm-suite-id = AlgorithmSuiteId(getEncryptionMaterialsRequest.algorithmSuite) +serialized-encryption-context = SerializeEncryptionContext(getEncryptionMaterialsRequest.encryptionContext) +NULL_BYTE = [0x00] + +ENTRY_ID = SHA384( + resource-id + + NULL_BYTE + + scope-id + + NULL_BYTE + + partition-id + + NULL_BYTE + + algorithm-suite-hex-byte + + NULL_BYTE + + algorithm-suite-id + + NULL_BYTE + + serialized-encryption-context ) ``` @@ -245,25 +358,56 @@ ENTRY_ID = SHA512( When the caching CMM receives a Decrypt Materials request, it MUST calculate the cache entry identifier as -the SHA-512 hash of the concatenation of the following byte strings, +the SHA-384 hash of the concatenation of the following byte strings, in the order listed: -1. The SHA-512 hash of a UTF-8 encoding of the caching CMM’s Partition ID -2. The two-byte algorithm suite ID corresponding to the algorithm suite in the request -3. The concatenation of the lexicographically-sorted SHA-512 hashes of the serialized encrypted data keys, +- MUST be the Resource ID for the Caching CMM (0x01) +- MUST be the Scope ID for Decrypt (0x02) +- MUST be the UTF-8 encoding of the caching CMM’s Partition ID +- Resource Suffix + - MUST be the two-byte Algorithm Suite ID corresponding to the algorithm suite in the request + - MUST be the concatenation of the lexicographically-sorted serialized encrypted data keys, where serialization is as defined in the [Encrypted Data Key Entries specification](../data-format/message-header.md#encrypted-data-key-entries). -4. A sentinel field of 512 zero bits (or equivalently, 64 null bytes), indicating the end of the key hashes -5. The SHA-512 hash of the serialized encryption context + - MUST be the serialized encryption context + +All the above fields must be separated by a single NULL_BYTE `0x00`. + +| Field | Length (bytes) | Interpreted as | +| -------------------------- | -------------- | -------------- | +| Resource ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Scope ID | 1 | bytes | +| Null Byte | 1 | `0x00` | +| Partition ID | Variable | bytes | +| Null Byte | 1 | `0x00` | +| Algorithm Suite ID | 2 | bytes | +| Null Byte | 1 | `0x00` | +| Concatenated Sorted EDKs | variable | bytes | +| Null Byte | 1 | `0x00` | +| SerializeEncryptionContext | Variable | bytes | As a formula: ``` -EDK_HASHES = [SHA512(SerializeEncryptedDataKey(key)) for key in decryptMaterialsRequest.encryptedDataKeys] -ENTRY_ID = SHA512( - SHA512(UTF8Encode(cachingCMM.partitionId)) - + AlgorithmSuiteId(decryptMaterialsRequest.algorithmSuite) - + CONCATENATE(SORTED(EDK_HASHES)) - + PADDING_OF_512_ZERO_BITS - + SHA512(SerializeEncryptionContext(decryptMaterialsRequest.encryptionContext)) +resource-id = [0x01] +scope-id = [0x02] +partition-id = UTF8Encode(cachingCMM.partitionId) +algorithm-suite-id = AlgorithmSuiteId(decryptMaterialsRequest.algorithmSuite) +concatenated-sorted-edks = CONCATENATE(SORTED(EDK)) +serialized-encryption-context = SerializeEncryptionContext(decryptMaterialsRequest.encryptionContext) +NULL_BYTE = [0x00] + +ENTRY_ID = SHA384( + resource-id + + NULL_BYTE + + scope-id + + NULL_BYTE + + partition-id + + NULL_BYTE + + algorithm-suite-id + + NULL_BYTE + + concatenated-sorted-edks + + NULL_BYTE + + serialized-encryption-context ) ``` diff --git a/framework/cryptographic-materials-cache.md b/framework/cryptographic-materials-cache.md index 491a9d38..2f232309 100644 --- a/framework/cryptographic-materials-cache.md +++ b/framework/cryptographic-materials-cache.md @@ -28,12 +28,15 @@ ## Implementations -| Language | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation | -| ---------- | -------------------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| C | 0.1.0-preview | 0.1.0 | [cache.h](https://github.com/aws/aws-encryption-sdk-c/blob/master/include/aws/cryptosdk/cache.h) | -| Javascript | 0.1.0-preview | 0.1.0 | [cryptographic_materials_cache.ts](https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/cache-material/src/cryptographic_materials_cache.ts) | -| Python | 0.1.0-preview | 1.3.0 | [caches/base.py](https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/caches/base.py) | -| Java | 0.1.0-preview | 1.3.0 | [CryptoMaterialsCache.java](https://github.com/aws/aws-encryption-sdk-java/blob/master/src/main/java/com/amazonaws/encryptionsdk/caching/CryptoMaterialsCache.java) | +| Language | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation | +| ------------ | -------------------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| C | 0.1.0-preview | 0.1.0 | [cache.h](https://github.com/aws/aws-encryption-sdk-c/blob/master/include/aws/cryptosdk/cache.h) | +| Javascript | 0.1.0-preview | 0.1.0 | [cryptographic_materials_cache.ts](https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/cache-material/src/cryptographic_materials_cache.ts) | +| Python | 0.1.0-preview | 1.3.0 | [caches/base.py](https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/caches/base.py) | +| Java | 0.1.0-preview | 1.3.0 | [CryptoMaterialsCache.java](https://github.com/aws/aws-encryption-sdk-java/blob/master/src/main/java/com/amazonaws/encryptionsdk/caching/CryptoMaterialsCache.java) | +| Dafny | 0.5.2 | 1.0.0 | [ICryptographicMaterialsCache in AwsCryptographyMaterialProvidersTypes.dfy](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/Model/AwsCryptographyMaterialProvidersTypes.dfy#L858-L1022) | +| Java (Dafny) | 0.5.2 | 1.0.0 | [ICryptographicMaterialsCache.java](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/materialproviders/ICryptographicMaterialsCache.java) | +| .NET | 0.5.2 | 1.0.0 | [CryptographicMaterialsCacheBase.cs](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/net/Generated/AwsCryptographicMaterialProviders/CryptographicMaterialsCacheBase.cs) | ## Overview diff --git a/framework/local-cryptographic-materials-cache.md b/framework/local-cryptographic-materials-cache.md index cd9b8958..11f3df7f 100644 --- a/framework/local-cryptographic-materials-cache.md +++ b/framework/local-cryptographic-materials-cache.md @@ -17,12 +17,15 @@ ## Implementations -| Language | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation | -| ---------- | -------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| C | n/a | n/a | [local_cache.c](https://github.com/aws/aws-encryption-sdk-c/blob/master/source/local_cache.c) | -| Javascript | 0.1.0 | 0.1.0 | [get_local_cryptographic_materials_cache.ts](https://github.com/aws/aws-encryption-sdk-javascript/blob/master/modules/cache-material/src/get_local_cryptographic_materials_cache.ts) | -| Python | n/a | n/a | [caches/local.py](https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/caches/local.py) | -| Java | 0.1.0 | 1.3.0 | [LocalCryptoMaterialsCache.java](https://github.com/aws/aws-encryption-sdk-java/blob/master/src/main/java/com/amazonaws/encryptionsdk/caching/LocalCryptoMaterialsCache.java) | +| Language | Confirmed Compatible with Spec Version | Minimum Version Confirmed | Implementation | +| ------------ | -------------------------------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| C | n/a | n/a | [local_cache.c](https://github.com/aws/aws-encryption-sdk-c/blob/master/source/local_cache.c) | +| Javascript | 0.1.0 | 0.1.0 | [get_local_cryptographic_materials_cache.ts](https://github.com/aws/aws-encryption-sdk-javascript/blob/master/modules/cache-material/src/get_local_cryptographic_materials_cache.ts) | +| Python | n/a | n/a | [caches/local.py](https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/caches/local.py) | +| Java | 0.1.0 | 1.3.0 | [LocalCryptoMaterialsCache.java](https://github.com/aws/aws-encryption-sdk-java/blob/master/src/main/java/com/amazonaws/encryptionsdk/caching/LocalCryptoMaterialsCache.java) | +| Dafny | 0.2.0 | 1.0.0 | [LocalCMC.dfy](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/src/CMCs/LocalCMC.dfy) | +| Java (Dafny) | 0.2.0 | 1.0.0 | [CryptographicMaterialsCache.java](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/materialproviders/CryptographicMaterialsCache.java) | +| .NET | 0.2.0 | 1.0.0 | [CryptographicMaterialsCache.cs](https://github.com/aws/aws-cryptographic-material-providers-library/blob/main/AwsCryptographicMaterialProviders/runtimes/net/Generated/AwsCryptographicMaterialProviders/CryptographicMaterialsCache.cs) | ## Overview