We have reviewed the human readable HTML (HRH) as a participant in the EHR vendor support for CQL testing. Our overall assessment of the HRH is that it is not adequate to convey the logic of the measure in simple enough terms to affect a basic understanding of the measure or to conduct reviews of measures during development. The HRH artifacts associated with the CQL implementation are significantly less readable than similar artifacts from the current version of the MAT 4.x. We made this assessment by reviewing the fairly simple CMS 68 and the more complex measure CMS 22 in both the current MAT 4.x and CQL implementations.

Our company has been capable of fully executing the Simple Logic XML from the current MAT version 4.x for several years now. We have supported real time evaluation of eCQM measures at the point of care in our product for 2015, 2016 and 2017 logic for MU2 and 2017 QPP. We have also implemented about 60 additional former PQRS measures (now QPP) that are not currently implemented as eCQMs using MAT 4.x and we intend to implement the remaining QPP measures within the next few quarters (all 272 QPP measures in total). During measure development using MAT 4.x we frequently conduct in house reviews of measure logic using primarily the human readable HTML (HRH) in conjunction with the QPP measure documents (PDFs). Thus we are very familiar with the measure development process, the level of effort and expertise required of implementers, and the difficulties in creating error-free measure logic. The current MAT 4.x artifacts are very readable, particularly the HRH and Simple Logic XML.

The HRH artifacts associated with the CQL testing measures are nearly impossible to use as a review tool. The two main problems are:
1. The logic expressions are not intuitive, possibly because they are not fully declarative and because the logic does not read sequentially. This results in an artifact that is very difficult to parse by reviewers and measure developers alike.
2. The readable logic is more than twice the length of the existing MAT 4.x HRH, possibly because the underlying CQL/ELM is more complex than necessary for the development of eQCMs.

In contrast, the current MAT 4.x HRH is compact and relatively easy to understand.

Consider a simple comparison between CQL HRH and MAT 4.x HRH using CMS 68 and 22 as examples:

1. CMS 68 is a very simple measure, which in MAT 4.x HRH takes 16 lines to express, and most of those lines are merely AND/OR statements or population headers. The CQL HRH for this measure is at least 36 lines (when not expanding sub-expression references or counting empty lines). The MAT 4.x HRH expressions read sequentially for the most part whereas the CQL HRH is very difficult to parse. Our reviews typically involve measure developers, clinicians, knowledge-base developers, and user-interface design personnel. The subject of measure logic development and review using highly-readable MAT 4.x is already very difficult and requires a specialized skill set. With the current CQL HRH it would be very difficult to conduct effective reviews.

2. CMS 22 has a very complex numerator. The MAT 4.x HRH logic can be expressed in a compact, readable, sequential 83 lines of which 20 are AND/OR statements or population headers. The CQL HRH for this measure clocks in at 185 lines and spans several pages and this without expanding sub-expressions or counting empty lines. Further it does not read in a sequential fashion due to the underlying nature of the CQL (e.g., the many alias "defines").

As it stands, with this unnecessary complexity in the logic model and the human readable rendering, the MAT 5/CQL logic will require higher maintenance and result in a consequent higher level of errors in authored measures. This will likely result in increased maintenance cost to all parties involved from authoring to execution. Currently the HRH artifact is unusable for logic review, which leaves the CQL (the ELM is just too difficult to read). The problem with the CQL as a review artifact is it is a full-blown programming language, and therefore not suitable for review by non-developers, which leaves out many clinicians, knowledge-base and terminology specialists, content developers and so on. CQL would not be a viable alternative for our review meetings.

What can be done to fix this? I'm not sure anything can solve this as long as CQL/ELM is the basis. If the CQL/ELM can be expressed in a readable fashion then the underlying structure of CQL could easily be simplified as well, something we would highly recommend. As an alternative, we may have to develop our own simple declarative language and re-author the eCQMs, which would be regrettable. In our opinion, for the current purposes of the eCQMs and the current set of 172 QPP measures, the simple logic merely lacks a few enhancements that currently hamper development (e.g., set complement function, assignment, etc.). For those purposes (i.e., QPP measure development) the CQL is not well suited.