progress

awslabs · Jan 16, 2025 · ea916ab · ea916ab
1 parent d2e1d7f
commit ea916ab
Show file tree

Hide file tree

Showing 3 changed files with 383 additions and 0 deletions.
diff --git a/framework/test-vectors/esdk-test-vectors/README.md b/framework/test-vectors/esdk-test-vectors/README.md
@@ -0,0 +1,111 @@
+[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved."
+[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0"
+
+## Summary
+
+These test vectors are intended to be used to validate interoperability
+across implementations of clients in a runtime,
+for the AWS Crypto Tools AWS Encryption SDK (ESDK).
+
+They are composed of JSON manifest files that define one or more test cases,
+including sufficient information to process each test case.
+These manifest files can also identify additional resources needed for a given test case.
+These resources will be identified with a URI.
+If identifying a local file, the URI will be a relative path from the manifest file's
+parent directory to the target file.
+
+Every manifest type definition must include a unique type name that will be used by
+test vector handlers to identify the manifest type.
+
+See https://github.com/awslabs/aws-crypto-tools-test-vector-framework for additional information.
+This framework is much of the motivation for this work.
+
+The manifest is meant to produce encryption and decryption scenarios that every
+runtime implementation can process and produce the desired output for that test vector.
+
+These test vectors are to be processed by the AWS Encryption SDK.
+
+```mermaid
+flowchart LR
+  subgraph ESDK Encryption-Scenario
+    direction LR
+    ESDK-Manifest --> ESDK-Java -->|ESDK-TestVector|EncryptionScenario
+    ESDK-Manifest --> ESDK-NET -->|ESDK-TestVector|EncryptionScenario
+    ESDK-Manifest --> ESDK-X -->|ESDK-TestVector|EncryptionScenario
+  end
+```
+
+```mermaid
+flowchart LR
+  subgraph MPL Decryption-Materials
+    direction LR
+    ESDK-Manifest+ESDK-TestVector --> ESDK-Java -->|ESDK-TestVector|DecryptionScenario
+    ESDK-Manifest+ESDK-TestVector --> ESDK-NET -->|ESDK-TestVector|DecryptionScenario
+    ESDK-Manifest+ESDK-TestVector --> ESDK-X -->|ESDK-TestVector|DecryptionScenario
+  end
+```
+
+```mermaid
+flowchart LR
+  subgraph ESDK Usage of MPL Manifest
+    direction LR
+    ESDK-EncryptKeyDescription-. Keyring-Construction .-> ESDK-Java --> ESDK-Manifest
+
+    ESDK-EncryptKeyDescription-. Keyring-Construction .-> ESDK-NET --> ESDK-Manifest
+  end
+```
+
+## Glossary
+
+- **Test Vector** : Information about a single test case. Used to either process existing data
+  or create new data.
+- **Test Vector Manifest** : A document that describes one or more test vectors.
+
+## Out of Scope
+
+This file is not a definition or description of any specific type of test vector or manifest.
+
+## Motivation
+
+The AWS Crypto Tools team has built several tools that require multiple implementations.
+Interoperability between these implementations and runtimes is critical.
+To ensure that these implementations are actually interoperable,
+we needed to define test vectors that would allow validation
+of various aspects of these tools.
+As we built more tools and required more types of test vectors,
+it became evident that there would be value in defining an extensible framework
+for defining many different types of test vectors to avoid having to reinvent the wheel
+for every client.
+
+We also need to be able to maintain backwards compatibility.
+By producing manifests that can be stored
+we can ensure that new clients can decrypt old messages.
+
+The latest version of the ESDK is written in Dafny
+and much of the design requirements are proven by Dafny specifications.
+These specifications are valuable but having empirical tests
+makes this proof even stronger.
+Because you can write and prove a specification
+but it may not be correct.
+This is especially true for backwards compatibility.
+But even for code reviews and understanding the impact of a change,
+looking at how test vectors are created helps reason about the edge cases of a change.
+
+## Drawbacks
+
+Not every configuration can be practically tested.
+See [test vector enumeration](test-vector-enumeration.md#selecting-a-representative-input-value) for details.
+
+We will need to write minimal clients in every language with which we want to use these test
+vectors to understand the manifests described in subsequent features.
+
+This should represent acceptable overhead: we would need to write some amount of code for each
+language to handle the test vectors anyway and this framework lets us define a consistent
+way of handling those test vectors while remaining simple to process.
+
+Additionally, using Dafny can lower this overhead.
+The challenge now is to compose the proper level of reusability.
+The ESDK is a top level client that consumes the MPL library. The ESDK
+should reuse the MPL to create the appropriate keyring. The top level 
+clients should focus on testing message format and client specific configurations
+and not focus on keyring creation.
diff --git a/framework/test-vectors/esdk-test-vectors/decryption-manifest.md b/framework/test-vectors/esdk-test-vectors/decryption-manifest.md
@@ -0,0 +1,74 @@
+[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved."
+[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0"
+
+# Decrypt Manifest
+
+## Version
+
+1.0.0
+
+## Dependencies
+
+This serves as a reference of all features that this feature depends on.
+
+| Feature                                                    | Min Version | Max Version |
+| ---------------------------------------------------------- | ----------- | ----------- |
+| [keys-description](../mpl-test-vectors/key-description.md) | 1           | n/a         |
+| [keys-manifest](../mpl-test-vectors/keys-manifest.md)      | 1           | n/a         |
+
+## Summary
+
+The decrypt manifest describes static test vectors.
+This include all the necessary details such as keyring, cmm, encryption context,
+algorithm suite, frame size, commitment policy, plaintext, etc.
+
+## Out of Scope
+
+This file is not a description of how the test vectors that will be generated by a compatible client should be decrypted.
+
+## Motivation
+
+We need a way of describing all scenarios that we want to cover with test vectors.
+This will be used for generating test vectors with an unknown implementation
+to be tested for compatibility using a known correct implementation.
+It will also help us reason about the correctness of specific features or keyrings.
+It can also be used to ensure known bad vectors or incorrect configuration cannot lead to successful decryption
+by demonstrating that different configurations succeed or fail correctly.
+
+## Guide-level Explanation
+
+This type of manifest describes test vectors to create.
+Processing these test scenarios will result in a [Decrypt Output](../../../client-apis/decrypt.md#output).
+
+Each decryption test scenario includes all necessary instructions to construct a
+[decrypt input request](../../../client-apis/decrypt.md#input).
+This includes all necessary inputs including algorithm suite, encryption context, frame size, commitment policy.
+
+## Reference-level Explanation
+
+### Manifest
+
+Map identifying the manifest.
+
+- `type` : Identifies the manifest as an AWS Encryption SDK message encryption manifest.
+  - MUST be `awses-encrypt`
+- `version` : Identifies the version of this feature document that describes the manifest.
+
+### Client
+
+Map identifying the client
+- `name`: Identifies the client that produced this AWS Encryption SDK message encryption manifest.
+  - MUST begin with `aws-encryption-sdk`
+  - MUST be suffixed with the runtime language that produced the encryption manifest
+- `version`: Identifies the version of the client that produced the encryption manifest.
+
+### Keys
+
+URI identifying a [keys manifest](./keys-manifest.md) to use with all tests.
+
+### Tests
+
+Map object mapping a test case ID to a test case description
+that describes how to generate a [decrypt input request](../../../client-apis/decrypt.md#input).
+Optional members on [decrypt input requests](../../../client-apis/decrypt.md#input)
+are optional in the test.
diff --git a/framework/test-vectors/esdk-test-vectors/encryption-manifest.md b/framework/test-vectors/esdk-test-vectors/encryption-manifest.md
@@ -0,0 +1,198 @@
+[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved."
+[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0"
+
+# Encrypt Manifest
+
+## Version
+
+1.0.0
+
+## Dependencies
+
+This serves as a reference of all features that this feature depends on.
+
+| Feature                                         | Min Version | Max Version |
+| ----------------------------------------------- | ----------- | ----------- |
+| [keys-description](./key-description.md)        | 1           | n/a         |
+| [keys-manifest](./keys-manifest.md)             | 1           | n/a         |
+| [encryption-manifest](../mpl-test-vectors/encryption-manifest.md) | 1           | n/a         |
+
+## Summary
+
+The encrypt manifest describes static test vectors.
+This include all the necessary details such as keyring, cmm, encryption context,
+algorithm suite, frame size, commitment policy, plaintext, etc.
+
+## Out of scope
+
+This file is not a description of how the test vectors that will be generated by a compatible client should be decrypted.
+
+## Motivation
+
+We need a way of describing all scenarios that we want to cover with test vectors.
+This will be used for generating test vectors with an unknown implementation
+to be tested for compatibility using a known correct implementation.
+It will also help us reason about the correctness of specific features or keyrings
+by demonstrating that different configurations succeed or fail correctly.
+
+## Guide-level Explanation
+
+This type of manifest describes test vectors to create.
+Processing these test scenarios will result in a [Encrypted Message](../../../client-apis/encrypt.md#encrypted-message).
+
+Each encryption test scenario includes all necessary instructions to construct a
+[encrypt input request](../../../client-apis/encrypt.md#input).
+This includes all necessary inputs including algorithm suite, encryption context, frame size, commitment policy.
+
+## Reference-level Explanation
+
+### Manifest
+
+Map identifying the manifest.
+
+- `type` : Identifies the manifest as an AWS Encryption SDK message encryption manifest.
+  - MUST be `awses-encrypt`
+- `version` : Identifies the version of this feature document that describes the manifest.
+
+### Client
+
+Map identifying the client
+- `name`: Identifies the client that produced this AWS Encryption SDK message encryption manifest.
+  - MUST begin with `aws-encryption-sdk`
+  - MUST be suffixed with the runtime language that produced the encryption manifest
+- `version`: Identifies the version of the client that produced the encryption manifest.
+
+### Keys
+
+URI identifying a [keys manifest](./keys-manifest.md) to use with all tests.
+
+### Plaintexts
+
+Map identifying the plaintext sizes supported in this manifest.
+
+These values MUST correspond to the [representative plaintext constraints](./esdk-test-vector-enumeration.md#representative-plaintext-constraints)
+
+### Tests
+
+Map object mapping a test case ID to a test case description
+that describes how to generate an [encrypt input request](../../../client-apis/encrypt.md#input).
+Optional members on [encrypt input request](../../../client-apis/encrypt.md#input)
+are optional in the test.
+
+- encryption-scenario : Encryption test vector description
+  - type : The type of test
+    - Allowed values
+      - positive-esdk : A successful request for encrypt input request that should also result in a successful decrypt request
+      - negative-encrypt-esdk : A encrypt request that should fail
+  - plaintext : The plaintext size to use for this encryption scenario
+  - description : A human readable description of what is being tested
+  - algorithmSuiteId: : Hex string of supported [Algorithm ID](../../algorithm-suites.md#algorithm-suite-id)
+  - frame-size: The frame size to use for this encryption-scenario
+  - encryptKeyDescription : A [key description](../mpl-test-vectors/key-description.md) used to request encrypt materials
+    This value will only exist types `positive-esdk` and `negative-decrypt-esdk` because these encrypt materials are expected to succeed
+- encryption-context: Map of keys and values to use for encryption context
+- reproduced-encryption-context: The reproduced encryption context to use in the decryption-scenario
+
+### Examples
+
+```json
+{
+  "tests": {
+    "d3e3d9f0-7ef7-4a3b-8e4a-f6ac9e2eda79": {
+      "encryption-scenario": {
+        "type": "positive-esdk",
+        "plaintext": "small",
+        "description": "Generated Discovery KMS MRK us-east-1-mrk->Filter aws 658956600833",
+        "algorithmSuiteId": "0014",
+        "frame-size": 512,
+        "encryptKeyDescription": {
+          "type": "aws-kms-mrk-aware",
+          "key": "us-east-1-mrk"
+        },
+        "decryptKeyDescription": {
+          "type": "aws-kms-mrk-aware-discovery",
+          "default-mrk-region": "us-west-2",
+          "aws-kms-discovery-filter": {
+            "partition": "aws",
+            "account-ids": [
+              "658956600833"
+            ]
+          }
+        },
+        "encryption-context": {},
+        "reproduced-encryption-context": {}
+      }
+    },
+    "65f83d70-c35b-4c50-8b29-3003177d3012": {
+      "encryption-scenario": {
+        "type": "positive-esdk",
+        "plaintext": "small",
+        "description": "Success testing requiredEncryptionContextKeys/reproducedEncryptionContext",
+        "algorithmSuiteId": "0478",
+        "frame-size": 512,
+        "encryptKeyDescription": {
+          "type": "required-encryption-context-cmm",
+          "underlying": {
+            "type": "raw",
+            "key": "aes-256",
+            "provider-id": "aws-raw-vectors-persistent-aes-256",
+            "encryption-algorithm": "aes"
+          },
+          "requiredEncryptionContextKeys": [
+            "b"
+          ]
+        },
+        "decryptKeyDescription": {
+          "type": "required-encryption-context-cmm",
+          "underlying": {
+            "type": "raw",
+            "key": "aes-256",
+            "provider-id": "aws-raw-vectors-persistent-aes-256",
+            "encryption-algorithm": "aes"
+          },
+          "requiredEncryptionContextKeys": [
+            "b"
+          ]
+        },
+        "encryption-context": {
+          "a": "a",
+          "b": "b"
+        },
+        "reproduced-encryption-context": {
+          "a": "a",
+          "b": "b"
+        }
+      }
+    },
+    "0b12c4b4-8a52-41b2-8e03-0a14ebf28e6f": {
+      "encryption-scenario": {
+        "type": "positive-esdk",
+        "plaintext": "small",
+        "description": "AWS KMS ECDH Discovery us-west-2-256-ecc discovery",
+        "algorithmSuiteId": "0214",
+        "frame-size": 512,
+        "encryptKeyDescription": {
+          "type": "aws-kms-ecdh",
+          "sender": "us-west-2-256-ecc",
+          "recipient": "us-west-2-256-ecc",
+          "sender-public-key": "sender-material-public-key",
+          "recipient-public-key": "recipient-material-public-key",
+          "ecc-curve": "us-west-2-256-ecc",
+          "schema": "static"
+        },
+        "decryptKeyDescription": {
+          "type": "aws-kms-ecdh",
+          "sender": "discovery",
+          "recipient": "us-west-2-256-ecc",
+          "sender-public-key": "discovery",
+          "recipient-public-key": "recipient-material-public-key",
+          "ecc-curve": "us-west-2-256-ecc",
+          "schema": "discovery"
+        },
+        "encryption-context": {},
+        "reproduced-encryption-context": {}
+      }
+    },
+  }
+}
+```