diff --git a/framework/test-vectors/esdk-test-vectors/README.md b/framework/test-vectors/esdk-test-vectors/README.md new file mode 100644 index 00000000..4e5dae5d --- /dev/null +++ b/framework/test-vectors/esdk-test-vectors/README.md @@ -0,0 +1,111 @@ +[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved." +[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0" + +## Summary + +These test vectors are intended to be used to validate interoperability +across implementations of clients in a runtime, +for the AWS Crypto Tools AWS Encryption SDK (ESDK). + +They are composed of JSON manifest files that define one or more test cases, +including sufficient information to process each test case. +These manifest files can also identify additional resources needed for a given test case. +These resources will be identified with a URI. +If identifying a local file, the URI will be a relative path from the manifest file's +parent directory to the target file. + +Every manifest type definition must include a unique type name that will be used by +test vector handlers to identify the manifest type. + +See https://github.com/awslabs/aws-crypto-tools-test-vector-framework for additional information. +This framework is much of the motivation for this work. + +The manifest is meant to produce encryption and decryption scenarios that every +runtime implementation can process and produce the desired output for that test vector. + +These test vectors are to be processed by the AWS Encryption SDK. + +```mermaid +flowchart LR + subgraph ESDK Encryption-Scenario + direction LR + ESDK-Manifest --> ESDK-Java -->|ESDK-TestVector|EncryptionScenario + ESDK-Manifest --> ESDK-NET -->|ESDK-TestVector|EncryptionScenario + ESDK-Manifest --> ESDK-X -->|ESDK-TestVector|EncryptionScenario + end +``` + +```mermaid +flowchart LR + subgraph MPL Decryption-Materials + direction LR + ESDK-Manifest+ESDK-TestVector --> ESDK-Java -->|ESDK-TestVector|DecryptionScenario + ESDK-Manifest+ESDK-TestVector --> ESDK-NET -->|ESDK-TestVector|DecryptionScenario + ESDK-Manifest+ESDK-TestVector --> ESDK-X -->|ESDK-TestVector|DecryptionScenario + end +``` + +```mermaid +flowchart LR + subgraph ESDK Usage of MPL Manifest + direction LR + ESDK-EncryptKeyDescription-. Keyring-Construction .-> ESDK-Java --> ESDK-Manifest + + ESDK-EncryptKeyDescription-. Keyring-Construction .-> ESDK-NET --> ESDK-Manifest + end +``` + +## Glossary + +- **Test Vector** : Information about a single test case. Used to either process existing data + or create new data. +- **Test Vector Manifest** : A document that describes one or more test vectors. + +## Out of Scope + +This file is not a definition or description of any specific type of test vector or manifest. + +## Motivation + +The AWS Crypto Tools team has built several tools that require multiple implementations. +Interoperability between these implementations and runtimes is critical. +To ensure that these implementations are actually interoperable, +we needed to define test vectors that would allow validation +of various aspects of these tools. +As we built more tools and required more types of test vectors, +it became evident that there would be value in defining an extensible framework +for defining many different types of test vectors to avoid having to reinvent the wheel +for every client. + +We also need to be able to maintain backwards compatibility. +By producing manifests that can be stored +we can ensure that new clients can decrypt old messages. + +The latest version of the ESDK is written in Dafny +and much of the design requirements are proven by Dafny specifications. +These specifications are valuable but having empirical tests +makes this proof even stronger. +Because you can write and prove a specification +but it may not be correct. +This is especially true for backwards compatibility. +But even for code reviews and understanding the impact of a change, +looking at how test vectors are created helps reason about the edge cases of a change. + +## Drawbacks + +Not every configuration can be practically tested. +See [test vector enumeration](test-vector-enumeration.md#selecting-a-representative-input-value) for details. + +We will need to write minimal clients in every language with which we want to use these test +vectors to understand the manifests described in subsequent features. + +This should represent acceptable overhead: we would need to write some amount of code for each +language to handle the test vectors anyway and this framework lets us define a consistent +way of handling those test vectors while remaining simple to process. + +Additionally, using Dafny can lower this overhead. +The challenge now is to compose the proper level of reusability. +The ESDK is a top level client that consumes the MPL library. The ESDK +should reuse the MPL to create the appropriate keyring. The top level +clients should focus on testing message format and client specific configurations +and not focus on keyring creation. diff --git a/framework/test-vectors/esdk-test-vectors/decryption-manifest.md b/framework/test-vectors/esdk-test-vectors/decryption-manifest.md new file mode 100644 index 00000000..ac5264fa --- /dev/null +++ b/framework/test-vectors/esdk-test-vectors/decryption-manifest.md @@ -0,0 +1,74 @@ +[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved." +[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0" + +# Decrypt Manifest + +## Version + +1.0.0 + +## Dependencies + +This serves as a reference of all features that this feature depends on. + +| Feature | Min Version | Max Version | +| ---------------------------------------------------------- | ----------- | ----------- | +| [keys-description](../mpl-test-vectors/key-description.md) | 1 | n/a | +| [keys-manifest](../mpl-test-vectors/keys-manifest.md) | 1 | n/a | + +## Summary + +The decrypt manifest describes static test vectors. +This include all the necessary details such as keyring, cmm, encryption context, +algorithm suite, frame size, commitment policy, plaintext, etc. + +## Out of Scope + +This file is not a description of how the test vectors that will be generated by a compatible client should be decrypted. + +## Motivation + +We need a way of describing all scenarios that we want to cover with test vectors. +This will be used for generating test vectors with an unknown implementation +to be tested for compatibility using a known correct implementation. +It will also help us reason about the correctness of specific features or keyrings. +It can also be used to ensure known bad vectors or incorrect configuration cannot lead to successful decryption +by demonstrating that different configurations succeed or fail correctly. + +## Guide-level Explanation + +This type of manifest describes test vectors to create. +Processing these test scenarios will result in a [Decrypt Output](../../../client-apis/decrypt.md#output). + +Each decryption test scenario includes all necessary instructions to construct a +[decrypt input request](../../../client-apis/decrypt.md#input). +This includes all necessary inputs including algorithm suite, encryption context, frame size, commitment policy. + +## Reference-level Explanation + +### Manifest + +Map identifying the manifest. + +- `type` : Identifies the manifest as an AWS Encryption SDK message encryption manifest. + - MUST be `awses-encrypt` +- `version` : Identifies the version of this feature document that describes the manifest. + +### Client + +Map identifying the client +- `name`: Identifies the client that produced this AWS Encryption SDK message encryption manifest. + - MUST begin with `aws-encryption-sdk` + - MUST be suffixed with the runtime language that produced the encryption manifest +- `version`: Identifies the version of the client that produced the encryption manifest. + +### Keys + +URI identifying a [keys manifest](./keys-manifest.md) to use with all tests. + +### Tests + +Map object mapping a test case ID to a test case description +that describes how to generate a [decrypt input request](../../../client-apis/decrypt.md#input). +Optional members on [decrypt input requests](../../../client-apis/decrypt.md#input) +are optional in the test. \ No newline at end of file diff --git a/framework/test-vectors/esdk-test-vectors/encryption-manifest.md b/framework/test-vectors/esdk-test-vectors/encryption-manifest.md new file mode 100644 index 00000000..1cf96b02 --- /dev/null +++ b/framework/test-vectors/esdk-test-vectors/encryption-manifest.md @@ -0,0 +1,198 @@ +[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved." +[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0" + +# Encrypt Manifest + +## Version + +1.0.0 + +## Dependencies + +This serves as a reference of all features that this feature depends on. + +| Feature | Min Version | Max Version | +| ----------------------------------------------- | ----------- | ----------- | +| [keys-description](./key-description.md) | 1 | n/a | +| [keys-manifest](./keys-manifest.md) | 1 | n/a | +| [encryption-manifest](../mpl-test-vectors/encryption-manifest.md) | 1 | n/a | + +## Summary + +The encrypt manifest describes static test vectors. +This include all the necessary details such as keyring, cmm, encryption context, +algorithm suite, frame size, commitment policy, plaintext, etc. + +## Out of scope + +This file is not a description of how the test vectors that will be generated by a compatible client should be decrypted. + +## Motivation + +We need a way of describing all scenarios that we want to cover with test vectors. +This will be used for generating test vectors with an unknown implementation +to be tested for compatibility using a known correct implementation. +It will also help us reason about the correctness of specific features or keyrings +by demonstrating that different configurations succeed or fail correctly. + +## Guide-level Explanation + +This type of manifest describes test vectors to create. +Processing these test scenarios will result in a [Encrypted Message](../../../client-apis/encrypt.md#encrypted-message). + +Each encryption test scenario includes all necessary instructions to construct a +[encrypt input request](../../../client-apis/encrypt.md#input). +This includes all necessary inputs including algorithm suite, encryption context, frame size, commitment policy. + +## Reference-level Explanation + +### Manifest + +Map identifying the manifest. + +- `type` : Identifies the manifest as an AWS Encryption SDK message encryption manifest. + - MUST be `awses-encrypt` +- `version` : Identifies the version of this feature document that describes the manifest. + +### Client + +Map identifying the client +- `name`: Identifies the client that produced this AWS Encryption SDK message encryption manifest. + - MUST begin with `aws-encryption-sdk` + - MUST be suffixed with the runtime language that produced the encryption manifest +- `version`: Identifies the version of the client that produced the encryption manifest. + +### Keys + +URI identifying a [keys manifest](./keys-manifest.md) to use with all tests. + +### Plaintexts + +Map identifying the plaintext sizes supported in this manifest. + +These values MUST correspond to the [representative plaintext constraints](./esdk-test-vector-enumeration.md#representative-plaintext-constraints) + +### Tests + +Map object mapping a test case ID to a test case description +that describes how to generate an [encrypt input request](../../../client-apis/encrypt.md#input). +Optional members on [encrypt input request](../../../client-apis/encrypt.md#input) +are optional in the test. + +- encryption-scenario : Encryption test vector description + - type : The type of test + - Allowed values + - positive-esdk : A successful request for encrypt input request that should also result in a successful decrypt request + - negative-encrypt-esdk : A encrypt request that should fail + - plaintext : The plaintext size to use for this encryption scenario + - description : A human readable description of what is being tested + - algorithmSuiteId: : Hex string of supported [Algorithm ID](../../algorithm-suites.md#algorithm-suite-id) + - frame-size: The frame size to use for this encryption-scenario + - encryptKeyDescription : A [key description](../mpl-test-vectors/key-description.md) used to request encrypt materials + This value will only exist types `positive-esdk` and `negative-decrypt-esdk` because these encrypt materials are expected to succeed +- encryption-context: Map of keys and values to use for encryption context +- reproduced-encryption-context: The reproduced encryption context to use in the decryption-scenario + +### Examples + +```json +{ + "tests": { + "d3e3d9f0-7ef7-4a3b-8e4a-f6ac9e2eda79": { + "encryption-scenario": { + "type": "positive-esdk", + "plaintext": "small", + "description": "Generated Discovery KMS MRK us-east-1-mrk->Filter aws 658956600833", + "algorithmSuiteId": "0014", + "frame-size": 512, + "encryptKeyDescription": { + "type": "aws-kms-mrk-aware", + "key": "us-east-1-mrk" + }, + "decryptKeyDescription": { + "type": "aws-kms-mrk-aware-discovery", + "default-mrk-region": "us-west-2", + "aws-kms-discovery-filter": { + "partition": "aws", + "account-ids": [ + "658956600833" + ] + } + }, + "encryption-context": {}, + "reproduced-encryption-context": {} + } + }, + "65f83d70-c35b-4c50-8b29-3003177d3012": { + "encryption-scenario": { + "type": "positive-esdk", + "plaintext": "small", + "description": "Success testing requiredEncryptionContextKeys/reproducedEncryptionContext", + "algorithmSuiteId": "0478", + "frame-size": 512, + "encryptKeyDescription": { + "type": "required-encryption-context-cmm", + "underlying": { + "type": "raw", + "key": "aes-256", + "provider-id": "aws-raw-vectors-persistent-aes-256", + "encryption-algorithm": "aes" + }, + "requiredEncryptionContextKeys": [ + "b" + ] + }, + "decryptKeyDescription": { + "type": "required-encryption-context-cmm", + "underlying": { + "type": "raw", + "key": "aes-256", + "provider-id": "aws-raw-vectors-persistent-aes-256", + "encryption-algorithm": "aes" + }, + "requiredEncryptionContextKeys": [ + "b" + ] + }, + "encryption-context": { + "a": "a", + "b": "b" + }, + "reproduced-encryption-context": { + "a": "a", + "b": "b" + } + } + }, + "0b12c4b4-8a52-41b2-8e03-0a14ebf28e6f": { + "encryption-scenario": { + "type": "positive-esdk", + "plaintext": "small", + "description": "AWS KMS ECDH Discovery us-west-2-256-ecc discovery", + "algorithmSuiteId": "0214", + "frame-size": 512, + "encryptKeyDescription": { + "type": "aws-kms-ecdh", + "sender": "us-west-2-256-ecc", + "recipient": "us-west-2-256-ecc", + "sender-public-key": "sender-material-public-key", + "recipient-public-key": "recipient-material-public-key", + "ecc-curve": "us-west-2-256-ecc", + "schema": "static" + }, + "decryptKeyDescription": { + "type": "aws-kms-ecdh", + "sender": "discovery", + "recipient": "us-west-2-256-ecc", + "sender-public-key": "discovery", + "recipient-public-key": "recipient-material-public-key", + "ecc-curve": "us-west-2-256-ecc", + "schema": "discovery" + }, + "encryption-context": {}, + "reproduced-encryption-context": {} + } + }, + } +} +```