Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test case permutation names can be confusing since they don't always have the same elements #887

Open
jhump opened this issue Jun 14, 2024 · 0 comments

Comments

@jhump
Copy link
Member

jhump commented Jun 14, 2024

Currently, if a test suite for example only applies to the Connect protocol, then there will be no Protocol:PROTOCOL_CONNECT element in the permutation name. The logic to create the permutation names attempts to make the names shorter by omitting elements that can't vary. But this can be counter-intuitive for users since it means the names are not always consistent. One would have to examine the test suite data and its properties to determine what the permutation names will look like.

Instead of favoring shorter permutation names, it might be nice if the names always included all properties. This can also make it easier to create patterns for a single protocol, compression, HTTP version, codec, or TLS setting. One could use a pattern like **/Protocol:PROTOCOL_CONNECT/** and be guaranteed that it matches all test case permutations that use the Connect protocol. Whereas today, there may be Connect-only test cases that do not contain that string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant