We have reviewed the CQL as a participant in the EHR vendor support for CQL testing. Our goal in this review was to assess the feasibility of using CQL in our commercial products to replace the current MAT 4.x simple logic XML for authoring and evaluating measure logic for the 272 QPP measures. Our company currently supports evaluation of MAT 4.x Simple Logic XML for the 2015, 2016, 2017 MU2 measures and about 110 QPP measures. Beyond the 53 eCQMs in the QPP we have authored about 60 additional QPP measures in the MAT 4.x and intend to implement all of the remaining QPP measures by year end using MAT 4.x artifacts.

In summary we find that CQL is probably not a viable approach given the open-ended nature of what appears to be a new fully-blown programming language. As a software vendor, we probably cannot devote the effort needed to create an interpreter, compiler or translator to enable evaluation of the entirety of the CQL/ELM. We also find that the CQL is not suitable as a replacement for the human readable artifact used in our reviews of measure logic. We have also found that the complete spectrum of capabilities of the CQL are not necessary to implement the 272 QPP measures. Supporting the full power of the CQL would probably be too expensive an undertaking.

Looking at the CQL we decided to review its potential use as a human readable source for review of measure logic, as an execution language and as a development language.

• Measure Review. Our conclusion is that the CQL is not suitable for review in comparison to the existing MAT 4.x artifacts. Yet, CQL would probably be a better source for review than the CQL-related human readable artifacts, due to the reasoning we supplied in CQLIT-39, titled "Review of CQL Human Readable Artifacts." However, the CQL is just not suitable for review by non-developers, which means the following members on our QPP development team would have a difficult time with the language in comparison to MAT 4.x and alternatives we can develop in-house: clinicians, knowledge-base and content developers, UI designers, testers, terminologists. So, CQL is not good for review in our opinion given the added effort and cost.

• Measure Execution. The CQL is not ideally suited for execution because it’s just another programming language with no execution support. Supporting CQL execution would require development of an interpreter or at least a method to translate CQL to a real programming language. Why develop measure logic in a language with no support rather than develop the measures in Java, C# or any other commercially supported programming language? A more sensible approach would have be to author the eCQMs in a commercially supported programming language using well-defined interfaces for the measure-specific constructs. Then each vendor merely has to implement these interfaces in their environment using whatever execution model they wish. One vendor might implement the “query” constructs by executing SQL whereas another might operate on a chart, in memory, at the point of care. As an aside, in our case, we have no intention of executing SQL during the evaluation of measures, however it appears that CQL is structured with that execution model in mind (another problem we see with CQL). I suspect if CQL supplants the Simple Logic XML of MAT 4.x for use in QPP eCQMs, that most EHRs and related software vendors will re-implement the measures anyway in their real software language of choice rather than execute CQL. So CQL is not a viable language for execution in our opinion. We understand that ELM is the execution model and there are tools to convert CQL to ELM. However, ELM is not something we can currently execute either, so really does not help. The same questions arise for ELM as an execution model as CQL. Why not just implement the measures directly in Java/C# with well-defined interfaces/APIs and be done with it, allowing each vendor to define their general implementation of the measure-specific language constructs.

• Measure Development. Then perhaps CQL is a viable language for authoring measures and some sort of translator can be developed to translate CQL into something manageable for execution? One problem we see with development using CQL is that our current measure developers must now become programmers. Again the same question arises. Why devote the effort to program in an unsupported standards language such as CQL rather than simply implement each measure individually in the real programming language of our choice? The required skill sets are basically the same. The effort is going to be about the same. So we are left with having to decide whether we create a translator for CQL, to translate into Java, C#, etc., or we simply redevelop those measures directly in Java/C#.

A standard such as CQL that is just an unsupported programming language is not useful for us. Does CQL really need to support libraries/file imports, importing of other files, structured values (tuples), list operations and other fully-blown programming language constructs in order to implement the current eCQMs and QPP measures? I have my doubts. Beyond the obvious, which is scrapping CQL entirely and going with in-house developed tools, we recommend a few things that might inspire us to commit to using something as open-ended as CQL:

1. The language should be pared back to a subset of the current capabilities, for possible release in stages, starting with only those features necessary for development of the existing measures. Or CMS could commit to using only a certain manageable set of constructs in its implementation.

2. A method should be provided to convert existing measures in MAT 4.x format to CQL.

3. A commitment should be made to not replace CQL in a few years by yet another standard. The point being that if MAT 4.x is going to be replaced by something completely different in MAT 5, do we have a guarantee the same thing will not happen again in MAT 6? As vendors we cannot afford to continually redevelop products when the supporting tools and standard are not compatible. There was a rather significant hit when MAT 4.3 made significant changes to the underlying logic. We took that hit, but cannot do that again.

Although we may be forced to deal with CQL on some level, we will consider the following alternatives before using CQL (and the available tools) in its current state:

1. Extend our current content development scripting language to support the necessary measure-logic constructs and re-develop/convert the eCQMs.

2. Redevelop and improve the current MAT 4.x capabilities in an in-house tool. We believe the current MAT 4.x is suitable for implementation of all QPP measures with a few necessary changes (e.g., complement set function, assignment, etc.). It’s not ideal, but it beats starting from scratch.

3. Develop a set of programming interfaces for measure-specific programming constructs and re-implement each measure directly in Java or C#.