Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encoding Requirement Classes in Metanorma #2

Closed
manuelfuenmayor opened this issue Jul 29, 2021 · 39 comments
Closed

Encoding Requirement Classes in Metanorma #2

manuelfuenmayor opened this issue Jul 29, 2021 · 39 comments
Assignees
Labels
enhancement New feature or request

Comments

@manuelfuenmayor
Copy link

manuelfuenmayor commented Jul 29, 2021

In relation to #1

I'm not sure how I should encode this Requirement Class in Metanorma:

[[rc_ogc-process-description]]
[cols="1,4",width="90%"]
|===
2+|*Requirements Class*
2+|http://www.opengis.net/spec/ogcapi-processes-1/1.0/req/ogc-process-description
|Target type |Web API
|Dependency |<<rc_core,OGC API - Processes Core>>
|Dependency |<<JSON,JSON>>
|===

Taking into account the documentation, maybe it should be something like this?

[requirement,type="class",label="http://www.opengis.net/spec/ogcapi-processes-1/1.0/req/ogc-process-description"]
====
[requirement,type="Dependency",label="<<rc_core,OGC API - Processes Core>>"]
======
======

[requirement,type="Dependency",label="<<JSON,JSON>>"]
======
======
====
@manuelfuenmayor manuelfuenmayor added the question Further information is requested label Jul 29, 2021
@opoudjis
Copy link

No, although the documentation may not explain this well enough. "Target type" and "Dependency" in OGC requirement tables are not embedded requirements, they are requirement attributes, encoded with distinct attributes.

[requirement,type="class",label="http://www.opengis.net/spec/ogcapi-processes-1/1.0/req/ogc-process-description",obligation="requirement",subject="Web API"]
====
inherit:[<<rc_core,OGC API - Processes Core>>]
inherit:[<<JSON,JSON>>]
====

We clearly do need rendered examples of the OGC requirements in metanorma.org, to make this clear.

@manuelfuenmayor
Copy link
Author

@opoudjis, another question, for cases like this:

[[per_job-list_limit-default-minimum-maximum]]
[width="90%",cols="2,6a"]
|===
^|*Permission {counter:per-id}* |*/per/job-list/limit-default-minimum-maximum*
^|A |The values for `minimum`, `maximum` and `default` in requirement `/req/job-list/limit-definition` are only examples and MAY be changed.
|===

I'm doing this:

[[per_job-list_limit-default-minimum-maximum]]
[.permission,label="/per/job-list/limit-default-minimum-maximum"]
====
[.permission,label="A"]
=====
The values for `minimum`, `maximum` and `default` in requirement `/req/job-list/limit-definition` are only examples and MAY be changed.
=====
====

Is that ok?

@manuelfuenmayor
Copy link
Author

Another example:

[[rec_core_process-execute-sync-document-ref]]
[width="90%",cols="2,6a"]
|===
|*Recommendation {counter:rec-id}* |/rec/core/process-execute-sync-document-ref +
^|Conditions |. The <<sc_execution_mode,negotiated execution mode>> is synchronous, `response=document`, `transmissionMode=reference`
. The number of outputs requested is 1 or more.
. The requested output value is a simple scalar value.
^|A |For a simple scalar values servers SHOULD consider supporting links to `text/plain` files that contain the output values.
|===

I'm doing:

[[rec_core_process-execute-sync-document-ref]]
[.recommendation,label="/rec/core/process-execute-sync-document-ref"]
====
[.recommendation,label="Conditions"]
=====
. The <<sc_execution_mode,negotiated execution mode>> is synchronous, `response=document`, `transmissionMode=reference`
. The number of outputs requested is 1 or more.
. The requested output value is a simple scalar value.
=====

[.recommendation,label="A"]
=====
For a simple scalar values servers SHOULD consider supporting links to `text/plain` files that contain the output values.
=====
====

@manuelfuenmayor
Copy link
Author

Another question: how would I encode Conformance Class blocks?

[[ats_callback]]
[cols="1,4",width="90%"]
|===
2+|*Conformance Class*
2+|http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback
|Target type |Web API
|Requirements class |<<rc_core,Requirements Class "Core">>
|===

UPDATE

Ok, now I know that I have to set type to "conformanceclass":

[[ats_callback]]
[requirement,type="conformanceclass",label="http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback",subject="Web API"]
====
...
====

But how to encode the "Requirements class" part?

@manuelfuenmayor
Copy link
Author

Also, in regard to Abstract test blocks, I'm passing from this:

[[ats_callback_job-callback]]
[width="90%",cols="2,6a"]
|===
^|*Abstract Test {counter:ats-id}* |*/conf/callback/job-callback*
^|Test Purpose |Validate the passing of a subscriber-URL in an execute request.
^|Requirement |<<req_callback_job-callback,/req/callback/job-callback>>
^|Test Method |. Configure a URL endpoint to accept message body from the server.
. Create an asynchronous execute request that includes the optional `subscriber` key (see https://raw.githubusercontent.com/opengeospatial/ogcapi-processes/master/core/openapi/schemas/execute.yaml[execute.yaml].
. Execute the asynchronous job using test <<ats_core_job-creation-op,/conf/core/job-creation-request>>.
. Validate the job results are received by the specified callback URL.
|===

to this:

[[ats_callback_job-callback]]
[requirement,type="abstracttest",label="/conf/callback/job-callback"]
====
[requirement,type="abstracttest",label="Test Purpose"]
======
Validate the passing of a subscriber-URL in an execute request.
======
[requirement,type="abstracttest",label="Requirement"]
======
<<req_callback_job-callback,/req/callback/job-callback>>
======
[requirement,type="abstracttest",label="Test Method"]
======
. Configure a URL endpoint to accept message body from the server.
. Create an asynchronous execute request that includes the optional `subscriber` key (see https://raw.githubusercontent.com/opengeospatial/ogcapi-processes/master/core/openapi/schemas/execute.yaml[execute.yaml].
. Execute the asynchronous job using test <<ats_core_job-creation-op,/conf/core/job-creation-request>>.
. Validate the job results are received by the specified callback URL.
======
====

Is it ok to set Test Purpose, Requirement and Test Method as labels?

@opoudjis
Copy link

opoudjis commented Jul 30, 2021

Ok. I am going to describe here what I have implemented for rendering. If there is new stuff in the spec you are encoding that I have not implemented, it is because I have not seen it or been asked to implement it until now, and I am not going to implement open-ended requirement encodings: if OGC want to see these, they must provide us with a more comprehensive descriptions of requirements that encompass them.

Which means, the answer for some of your questions is going to be, I don't know, we have not covered that yet, and that requires further input from OGC.

A recommendation under OGC is rendered as follows:

  • It is a table
  • The CSS class of the recommendation table is the type attribute of the recommendation. The only types recognised are: verification, abstracttest, class, conformanceclass, and recommend.
  • The heading of the table (spanning two columns) is its name (the Asciidoctor role or style of the recommendation, e.g. [.permission] or [permission]), optionally followed by its title (the caption of the recommendation, e.g. .Title).
  • The title of the table (spanning two columns) is its label attribute.
  • The initial rows of the body of the table are:
    • The obligation attribute of the recommendation, if given: Obligation followed by the attribute value
    • The subject attribute of the recommendation, if given: Target Type (for Class or Conformance Class requirement) or Subject, followed by the attribute value
    • The inherit attribute of the recommendation, if given: Dependency followed by the attribute value
    • The classification attributes of the recommendation, if given: the classification tag (in capitals), followed by the classification value.
  • The remaining rows of the recommendation are the components of the recommendation, encoded as table rows instead of as a definition table (as they are by default in Metanorma).
    • Components can include nested recommendations; these are expected in the class of Class and Conformance Class recommendations.
    • Components can include open blocks marked with role attributes; the Standoc default recognised components are [.specification], [.measurement-target], [.verification], [.import]. Other components need to be added explicitly in code.

opoudjis added a commit to metanorma/metanorma.org that referenced this issue Jul 30, 2021
@opoudjis
Copy link

So:

@opoudjis, another question, for cases like this:

[[per_job-list_limit-default-minimum-maximum]]
[width="90%",cols="2,6a"]
|===
^|*Permission {counter:per-id}* |*/per/job-list/limit-default-minimum-maximum*
^|A |The values for `minimum`, `maximum` and `default` in requirement `/req/job-list/limit-definition` are only examples and MAY be changed.
|===

I'm doing this:

[[per_job-list_limit-default-minimum-maximum]]
[.permission,label="/per/job-list/limit-default-minimum-maximum"]
====
[.permission,label="A"]
=====
The values for `minimum`, `maximum` and `default` in requirement `/req/job-list/limit-definition` are only examples and MAY be changed.
=====
====

Is that ok?

No. Whatever the "A" is doing there, it is not a label. The label spans across two columns.

^|*Permission {counter:per-id}* |*/per/job-list/limit-default-minimum-maximum* is clearly part of the table heading: "Permission #n" is taken from the name (role) of the requirement, [.permission]. We have made no provision for headings having two columns, because the examples we worked from did not have headings with two columns. Encoding label="/per/job-list/limit-default-minimum-maximum" is correct, it will just render as a row, not a second column.

But I have no idea what the "A" is supposed to be, and put there, it will not do anything useful. A nested requirement as you have it is only going to generate the requirement identifier and the requirement label, e.g. "Permission 1-1 | A"; it will NOT generate the permission content. Nested permissions are only intended for summary representation of permissions within a permission class.

Looking at

https://github.com/opengeospatial/ogcapi-processes/blob/91121176dbb3b4241bc221767fb3ae886c2216ca/core/recommendations/core/REC_job-status.adoc

it is very clear that there is stuff in the requirements of OGC that we have never been told of, and we have never put into the gem. We do not have conditions as a distinct component (though it might be equivalent to verification), and we do not have any notion of requirement clauses being labelled with letters (if that's what's going on).

The formatting distinction being made between

https://raw.githubusercontent.com/opengeospatial/ogcapi-processes/91121176dbb3b4241bc221767fb3ae886c2216ca/core/recommendations/core/REC_test-process.adoc

[[rec_core_test-process]]
[width="90%",cols="2,6a"]
|===
|Recommendation {counter:rec-id} |/rec/core/test-process +

If a server implementing the OGC API - Processes - Part 1: Core is tested using CITE tests, the server SHOULD offer one of the following options:

. An Echo process that returns any input that is provided, without any actual processing. 
. Provide example input data for a specific process.

The process logic SHOULD include a delay, whether through actual processing or a simple sleep mechanism, in order to test asynchronous execution.

|===

and

[[rec_core_next-1]]
[width="90%",cols="2,6a"]
|===
^|*Recommendation {counter:rec-id}* |*/rec/core/next-1* 
^|A |A `200`-response SHOULD include a link to the next "page" (relation: `next`), if more process summaries have been selected than returned in the response.
|===

where the first does not have a separate row and the second does, appears utterly random.

Metanorma does not do random. If this differentiation is to be preserved in Metanorma, someone is going to need to explain it to me.

@opoudjis
Copy link

Another question: how would I encode Conformance Class blocks?

[[ats_callback]]
[cols="1,4",width="90%"]
|===
2+|*Conformance Class*
2+|http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback
|Target type |Web API
|Requirements class |<<rc_core,Requirements Class "Core">>
|===

UPDATE

Ok, now I know that I have to set type to "conformanceclass":

[[ats_callback]]
[requirement,type="conformanceclass",label="http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback",subject="Web API"]
====
...
====

But how to encode the "Requirements class" part?

The type is just "class". Because it is a requirement, Metanorma will fill in the information that it is a Requirement Class on rendering.

@opoudjis
Copy link

opoudjis commented Jul 30, 2021

Is it ok to set Test Purpose, Requirement and Test Method as labels?

No. What these should be is:

[[ats_callback_job-callback]]
[requirement,type="abstracttest",label="/conf/callback/job-callback"]
====

[.test-purpose]
Validate the passing of a subscriber-URL in an execute request.

[.requirement]
<<req_callback_job-callback,/req/callback/job-callback>>

[.test-method]
-----
. Configure a URL endpoint to accept message body from the server.
. Create an asynchronous execute request that includes the optional `subscriber` key (see https://raw.githubusercontent.com/opengeospatial/ogcapi-processes/master/core/openapi/schemas/execute.yaml[execute.yaml].
. Execute the asynchronous job using test <<ats_core_job-creation-op,/conf/core/job-creation-request>>.
. Validate the job results are received by the specified callback URL.
----
====

And I cannot entertain a major refactor of how we process requirements right now, with my workload. I am certainly not going to entertain it without a specification from OGC of a controlled vocabulary of requirement components; and "Requirement", as a cross-reference to an existing requirement, is something I have not seen before either.

@manuelfuenmayor
Copy link
Author

No. Whatever the "A" is doing there, it is not a label. The label spans across two columns.

@opoudjis, there are cases where the word "conditions" appears:

[[rec_core_process-execute-sync-document-ref]]
[width="90%",cols="2,6a"]
|===
|*Recommendation {counter:rec-id}* |/rec/core/process-execute-sync-document-ref +
^|Conditions |. The <<sc_execution_mode,negotiated execution mode>> is synchronous, `response=document`, `transmissionMode=reference`
. The number of outputs requested is 1 or more.
. The requested output value is a simple scalar value.
^|A |For a simple scalar values servers SHOULD consider supporting links to `text/plain` files that contain the output values.
|===

Not a label as well?

@opoudjis
Copy link

Conditions, like test method and test purpose, are subcomponents of a requirement, which we have not seen before, and therefore will not recognise. I am trying to work out what a general solution looks like.

@manuelfuenmayor
Copy link
Author

Another question: how would I encode Conformance Class blocks?

[[ats_callback]]
[cols="1,4",width="90%"]
|===
2+|*Conformance Class*
2+|http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback
|Target type |Web API
|Requirements class |<<rc_core,Requirements Class "Core">>
|===

UPDATE
Ok, now I know that I have to set type to "conformanceclass":

[[ats_callback]]
[requirement,type="conformanceclass",label="http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback",subject="Web API"]
====
...
====

But how to encode the "Requirements class" part?

The type is just "class". Because it is a requirement, Metanorma will fill in the information that it is a Requirement Class on rendering.

What do you mean that the type is just "class"? Should I replace "conformanceclass" by "class" or do you refer to a nested block?

@opoudjis
Copy link

Another question: how would I encode Conformance Class blocks?

[[ats_callback]]
[cols="1,4",width="90%"]
|===
2+|*Conformance Class*
2+|http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback
|Target type |Web API
|Requirements class |<<rc_core,Requirements Class "Core">>
|===

UPDATE
Ok, now I know that I have to set type to "conformanceclass":

[[ats_callback]]
[requirement,type="conformanceclass",label="http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/callback",subject="Web API"]
====
...
====

But how to encode the "Requirements class" part?

The type is just "class". Because it is a requirement, Metanorma will fill in the information that it is a Requirement Class on rendering.

What do you mean that the type is just "class"? Should I replace "conformanceclass" by "class" or do you refer to a nested block?

The types we recognise are:

  • conformanceclass
  • class
  • abstracttest
  • verification

Requirement Class, Permission Class and Recommendation Class are all just "class" as far as we are concerned. conformanceclass is distinct from class.

@manuelfuenmayor
Copy link
Author

manuelfuenmayor commented Jul 30, 2021

Ok, but clearly we are in a conformanceclass case, right?

Here's another example:

[[ats_html]]
[cols="1,4",width="90%"]
|===
2+|*Conformance Class*
2+|http://www.opengis.net/spec/ogcapi-processes-1/1.0/conf/html
|Target type |Web API
|Requirements class |<<rc_html,Requirements Class "HTML">>
|Dependency |<<ats_core,Conformance Class 'Core'>>
|===

I mean, the text "Conformance Class" is present as header.
If I set the type to class, that text ("Conformance Class") will not be indicated in output, right?

@opoudjis
Copy link

Oh, now I understand.

We do not support that either.

@opoudjis opoudjis added enhancement New feature or request and removed question Further information is requested labels Jul 30, 2021
@opoudjis
Copy link

Manuel is going to keep tracking instances we don't cover here. The outcome of discussions here, and under opengeospatial#239, is that we will extend the coverage of requirements under OGC:

  • We will extend the classes of components recognised for requirements, to be either open-ended, or at minimum Condition, Test Method, Test Purpose, Part
  • The Metanorma XML representation of these will make the name of these components an attribute, rather than a tag name, to allow them to be extensible.
  • Parts will be autonumbered on rendering, with capital alphabetic numbering.
  • We will work out what the Requirement and Requirement Class cross-references are, and whether they are to be encoded distinct from Inherit and Dependency

@manuelfuenmayor
Copy link
Author

@opoudjis
In some Abstract Tests there is like an extension of the Test Method partition if certain condition is met.

For example in: https://raw.githubusercontent.com/metanorma/ogcapi-processes/master/core/abstract_tests/core/ATS_job-results-success-async-document.adoc

In my opinion, both the NOTE block and the table below of it should be inside the first table (or Abstract Test block).

@manuelfuenmayor
Copy link
Author

In addition, there are cases where a table (an Schema and Tests table) is added below the Abstract Test block.

For example: https://raw.githubusercontent.com/metanorma/ogcapi-processes/master/core/abstract_tests/core/ATS_job-results-exception-no-such-job.adoc

I'm not sure if this table should be part of the Abstract Test block.

@manuelfuenmayor
Copy link
Author

Ok, hopefully those were the last particular cases that (I think) we don't support.

ronaldtse added a commit to metanorma/metanorma.org that referenced this issue Aug 4, 2021
@opoudjis
Copy link

opoudjis commented Aug 12, 2021

Remaining tasks:

  • We will enforce the cardinalities of ModSpec. (Down the road, we can do much more validation to ensure compliance to ModSpec, but that will have to wait)
  • We will need to introduce cross-refererences to parts of requirements, which incorporate the incrementing letter
  • We will allow crossreferences as the values of subject and inherit
  • We will allow multiple values for subject
  • We will allow a definition list to contain metadata about recommendations, as an alternative to the unmanageably long attribute list, tagging it with the role [%metadata].
  • Verify the interpretation of ModSpec against Lutaml and Turtle

opoudjis added a commit to metanorma/metanorma.org that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/metanorma-ogc that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/metanorma-standoc that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/isodoc that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/metanorma-requirements-models that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/metanorma-requirements-models that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/metanorma.org that referenced this issue Aug 13, 2021
opoudjis added a commit to metanorma/metanorma-standoc that referenced this issue Aug 13, 2021
@opoudjis
Copy link

Remaining tasks:

  • We will enforce the cardinalities of ModSpec. (Down the road, we can do much more validation to ensure compliance to ModSpec, but that will have to wait)
    • I think I'll pass for now, and defer to Lutaml processing
  • Verify the interpretation of ModSpec against Lutaml and Turtle

@opoudjis
Copy link

Deferring validation to metanorma/metanorma-ogc#242

@anermina
Copy link

My suggestion: treat the text under the table as part of the requirement, but have it crossreference the table, and leave the table outside.

@opoudjis , I've recently made an adjustment in all cases of this kind and @manuel489 told me that you've previously agreed on applying markup in accordance to this comment of yours.
Do you agree to use following markup instead, or should I revert the changes?

. Validate the document for all supported media types using the resources and tests identified in <<job-results-exception-no-such-job>>
--
====

The job results page for a job may be retrieved in a number of different formats. The following table identifies the applicable schema document for each format and the test to be used to validate the job results for a non-existent job against that schema.  All supported formats should be exercised.

[[job-results-exception-no-such-job]]
.Schema and Tests for the Job Result for Non-existent Job

This is in regard to PR opengeospatial#254.
Thanks in advance!

ronaldtse pushed a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
ronaldtse added a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
ronaldtse pushed a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
ronaldtse pushed a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
ronaldtse pushed a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
ronaldtse pushed a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
ronaldtse pushed a commit to metanorma/metanorma.org that referenced this issue Oct 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants