Introduction
This guide serves as a reference for the Cancer Report Validator (CRV) and provides details regarding the user interface and how to interpret the results reported to users.
The Home tab can be accessed at the top of the screen and describes the purpose and background of the CRV. A link to the test data is also included.
Validator Tab
The Validator tab is displayed by default and can be accessed at the top of the screen.
Under the Submit section, the user should select a file that represents the cancer event report that should be validated.
There are two options for validation: context-free and context-specific.
Context-free (IG Conformance) Validation
In context-free validation, the submitted cancer event report is only validated according to the conformance statements in the Cancer Registries IG, without any comparison of the data element values to the test data.
- From the Test Cases dropdown menu, select the "No test case selected" option.
- Press the Upload button.
- The results will be displayed in the following tabs:
- Details
- IG Conformance Events.
Context-specific (Test Data Conformance) Validation
In context-specific validation, data element values in the submitted cancer event report are matched against those specified in the test data spreadsheets (see Test Data Documentation).
- From the Test Cases dropdown menu, select any of the test case options besides the "No test case selected" option to identify which test case values the submitted cancer event report will be matched against.
- Press the Upload button.
- The results will be displayed in the following tabs:
- Details
- IG Conformance Events
- Test Data Conformance Events.
As part of context-specific validation, the CRV also checks that certain pairs of values in the report are equal. This occurs as follows:
- Test Case 1 – Medications Administered Section
- The <id/@root> in the Indication template of the Medications Activity #1 (gemcitabine 38 MG/ML injectable solution) must be the same as the <id/@root> for the breast cancer Problem Observation (Malignant neoplasm of breast upper outer quadrant).
- Test Case 3 – Cancer Diagnosis Section
- The <id/@root> in the Reference Entry Relationship (CONF:1169-33192) of Cancer Diagnosis Observation # 1 (Lentigo Maligna/Hutchinson’s Melanotic Freckle) must be the same as the <id/@root> of Problem Observation #1 (Lentigo Maligna/Melanoma in situ of left upper limb/Malignant Melanoma of upper limb).
- The <id/@root> in the Reference Entry Relationship (CONF:1169-33192) of Cancer Diagnosis Observation # 2 (Nodular Melanoma) must be the same as the <id/@root> of Problem Observation #2 (Malignant melanoma of skin of scalp and neck).
- Test Case 3 – Procedures Section
- The <id/@root> in the Indication template of the Procedure Activity Procedure #1 (Excision of malignant neoplastic lesion of skin of extremities/Excision, malignant lesion including margins, trunk, arms, or leg/Excision of Left Upper Arm Skin) must be the same as the <id/@root> of Problem Observation #1 (Lentigo Maligna/Melanoma in situ of left upper limb/Malignant Melanoma of upper limb).
- The <id/@root> in the Indication template of the Procedure Activity Procedure #2 (Excision of malignant lesion of scalp and neck/Excision, malignant lesion including margins, scalp, neck, hands, feet, genitalia/Excision of Neck Skin, External Approach) must be the same as the <id/@root> of Problem Observation #2 (Malignant melanoma of skin of scalp and neck).
Test Case Details
Please see the Test Data Documentation for more details regarding the test cases.
Results
The results of each cancer event report submission are displayed in an accordion-style collapsible panel where a summary is displayed in a color-coded rectangular bar and more detailed results are displayed within the panel in several tabs. In the right-hand corner of the rectangular bar, the user can click on the X to remove the results for an entire submission.
Summary
The rectangular bar at the top of the panel lists the file name that was submitted, the time of submission, the test case that was selected, and a results summary.
- For Result (IG Conformance Events), the possible values are Pass or Fail. The rectangular bar is green if this result is "Pass" and is red otherwise.
- For Result (Test Data Conformance Events), the possible values are "No test data mismatches" or "Test data mismatches exist." Only present when a test case is selected from the Test Cases dropdown menu.
Downloading Results
The results can be downloaded in either JSON or XML format from the rectangular bar by clicking on either the JSON or XML icons in the "Download" box.
Details Tab
The Details tab includes information regarding the cancer event report submission and the test case that was selected.
The IG Conformance Events tab includes a summary of the IG conformance validation events above a table containing a row for each validation event.
Summary Statistics
The statistics represent the number of IG conformance validation events that fall into one of the following categories:
- All: total number
- Errors: represented by a SHALL conformance statement
- Warnings: represented by a SHOULD conformance statement
- Infos: represented by a MAY conformance statement
For each of the categories above, the following numbers are displayed: a of b (c Filtered), where:
- a = # events displayed of total
- b = total # in the category
- c = b - a
Table Results
Prior to any filtering of validation events in the table, the results for all IG conformance validation events are displayed. Any errors are displayed as the first rows of the table. The table below describes the columns in the table and the available filtering options on a per column basis, if applicable.
| Column Name | Description | Filtering Options |
|---|
| Status | Indicates the status of the validation event - Pass: the cancer event report submission conforms to the IG for the particular assertion being tested
- ERROR, WARN, or INFO: the level associated with the particular assertion being tested for which the submission doesn't conform to the IG
| ERROR, WARN, INFO, or Pass |
| Location | Indicates the general location in the submitted report where an element corresponding to the assertion tested can be found in terms of - Node Expression: an XPath expression
- Line #
- Column #
| Line number |
| Expression | - Context Expression: refers to the context of the Schematron rule in which the assertion corresponding to the validation event is run.
- Runtime Test Expression: refers to the assertion that is being tested, as written in the Schematron. If the assertion involves a code that needs to come from a specific value set and/or code system, information referencing static or dynamic vocabulary validation may also be present as part of this expression.
| |
| Description | Text associated with/describing the conformance statement as listed in the IG. | |
| Messages | Additional messages that the user may want to look at, which pertain to validation events that involve vocabulary validation. These messages could further explain the status and/or result of the validation event. e.g. "Vocabulary validation disabled for assertion," "Code was not specified," "Invalid code," "Invalid code display name." | |
| Vocabulary Sets | Includes information regarding the value set and/or code system from which the code was selected from (e.g. OIDs, name, version), if applicable. Displays the actual and expected results. | ID |
| Code | Applicable to events that contain some type of code attribute - ID: code value
- Name: display name
| ID |
| Pattern | A pattern in the Schematron file corresponding to a template id in the IG, including a phase level (errors, warnings, infos). Patterns are composed of rules. | ID |
| Rule | A rule in the Schematron file that is composed of assertions. | ID |
| Assertion | An assertion in the Schematron file corresponding to a conformance statement(s) in the IG. - ID: Has "a-" appended to a conformance statement ID in the IG, possibly followed by other characters (e.g. a-81-7154)
| ID |
| ID | Validation event id, which users can provide in support requests when they have questions regarding a particular assertion. The same validation id will correspond to the same assertion that is tested for a given cancer event report submission. | |
The Test Data Conformance Events tab includes a summary of the test data conformance validation events above a table containing a row for each validation event.
Summary Statistics
The statistics represent the number of test data conformance validation events that fall into one of the following categories:
- All: total number
- Test Data Mismatches: when the actual result does not match one of the expected results from the test data
For each of the categories above, the following numbers are displayed: a of b (c Filtered), where:
- a = # events displayed of total
- b = total # in the category
- c = b - a
Table Results
Prior to any filtering of validation events in the table, the results for all test data validation events are displayed. Any mismatches are displayed as the first rows of the table. The table below describes the columns in the table and the available filtering options on a per column basis, if applicable.
| Column Name | Description | Filtering Options |
|---|
| Status | Indicates the status of the validation event - Match: the value in the cancer event report submission for a data element matches the expected value specified in the test data
- Mismatch: the value in the cancer event report submission for a data element does not match the expected value specified in the test data
| Mismatch or Match |
| Actual Result | The value found in the submitted report at the location specified in the "Location" column. | ID |
| Expected Result(s) | The possible value(s) that could be used in the submitted report, as specified in the test data. | ID |
| Location | Indicates the location in the submitted report where a data element corresponding to the test data can be found in terms of- Node Expression: an XPath expression
- Line #
- Column #
| Line # |
| Messages | Additional messages that the user may want to look at, which further explain the status and/or result of the validation event (i.e. why there is a mismatch). Examples include: "Mismatch is part of a group of XPath expressions, in which one must be satisfied for the event to be considered a match." "Actual result matches expected result, but is not found in the expected location."
| |
| ID | Validation event id, which users can provide in support requests when they have questions regarding a particular test data match or mismatch. The same validation id will correspond to the same match or mismatch tested for a given cancer event report submission. | |