Monday, June 22, 2009

A New Approach to Certification

Last week I spoke with Mark Leavitt, the CEO of CCHIT, about his best thinking regarding certification in a post-ARRA world.

In the past there have been 3 groups who have requested improvements to existing certification criteria:

1. Self developers who achieve a high degree of functionality through continuous improvement of home built software

2. The Open Source community

3. The Health 2.0/iPhone as application platform/multiple thin web-application combined to provide EHR-Lite functionality community

CCHIT held 2 Town Halls, each with 500 people, to publicly discuss a new approach to certification.

Mark's slides are available online

The New path to certification has three branches:

EHR-C The Certification of a Comprehensive EHR. This is what has been done to date. The product itself is certified, not the specific implementation at a specific site. EHR-C products should be able to meet all meaningful use criteria if implemented properly. The cost to the vendor for certification is $30,000-50,000 per product. EHR-C is for providers who seek maximal assurance of EHR compliance and capabilities.

EHR-M The Certification of a Module such as e-prescribing, lab ordering/resulting, clinical charting and data exchange. A clinician could assemble multiple modules and be certified regarding the specific functions they perform. Integration of data between modules is the clinician's responsibility. Meaningful use may be possible, but depends upon how the modules are used together. The cost to the vendor for certification is $5000-$35,000 per module. EHR-M is for providers who prefer to integrate technologies from multiple certified sources.

EHR-S The Certification of Site for the functionality that can be achieved using the software installed there. A screen capture function is used to document capabilities and this capture will be reviewed by an expert remotely. The cost is $150-300 per licensed provider. EHR-S is for providers who self- develop or assemble EHRs from non-certified sources.

In addition to these three paths, “version lockdown” is no longer needed or relevant. For EHR-C and EHR-Mcertifications, updated or enhanced versions of a code base would inherit certified status without need for CCHIT approval. For EHR-S sites, updates or enhancements would not require recertification.

I've heard a great deal of positive feedback about this new approach. The work of CCHIT to enhance its approach, the work of the HIT Standards Committee to define the certification criteria needed for meaningful use, and the work of the HIT Policy Committee to review certification in general will result in a comprehensive certification plan aligned with ARRA by the end of of 2009.


Donald Green MD said...
This comment has been removed by the author.
Donald Green MD said...

Bravo. Finally some sense to this process. Small practices needs and organization differs so much from larger entities. Also the expense for licensing seems much more reasonable under this plan. It is also great news that once certified as a small user, license renewal will be unnecessary as long as the orginal evaluation showed hign functionality within the context of a smaller practice. This new approach to certification will certainly do much to bring about innovative emr products that will be useful to the medical world. Thanks for the update. As one of those small developers I greatly appreciate this change in direction. It is also finally a recognition that medical professionals can evaluate and bear the responsibility for the system they use. Thanks for the update.

Unknown said...

I tried explaining this at last Friday's HealthCampMaryland (twitter|video) and was met with skepticism.

The audience had a strong group of open source proponents who focused on this main issue: $5k for a module to be certified. Look at Drupal, which has hundreds of modules that can be plugged in to it. Now imagine a Drupal-like EHR, with dozens of open source modules that can be plugged in to provide different functionalities. They thought the pricing for this made it a non-starter.

I communicated my vision of how this marketplace might work, but I don't think I did a great job of it (or, as I suspect, there is an automatic attack mode that engages when the open source mind hears the term "CCHIT", thus disabling the auditory cortex). Here it is:

1-at $150-300/site implementation, this is the most inexpensive way to go.

2-a community of open sourcers bands together to develop an online marketplace community of EHR developers, users, integrators, and consultants -- like a SourceForge or

3-providers can shop for an EHR here, with varying levels of assistance, from free downloads and self-installs to full installation, maintenance, and support by an integrator at an initial and/or monthly price.

4-the secret sauce here would be the feeding back of information about site installations that received successful certification. Modules that were used in a certified implementation would note such. Developers, integrators, and consultants who helped to construct a successful implementation would likewise be indicated, verified by the provider, whose certified implementation would no doubt come with some sort of seal of certification by CCHIT.

Thus, one could sort full EHRs and modules by the successful number of certified implementations out there, and there would be lots of provider reviews, screenshots, ratings, discussions, etc, to round out the 2.0-ification of it all.

I think this has real potential. Think E-Bay+iTunes+Angie's List.


Claudio Luís Vera (@modulist) said...

The new proposed CCHIT certification standards are a small step in the right
direction. Unfortunately, they also betray CCHIT's profound lack of understanding of the economics and technology behind open source.

In order to work to its potential, an open source community would be comprised of thousands of independent developers. Developers work with existing open source such as VistA, or download an SDK. They then are free to improve it by building modules on top of the original. In order to make the time investment feasible to a single developer, these modules are typically very small in scope. Outside healthcare, an open source deployment normally consists of scores or hundreds of "bite-sized" modules running on top of a kernel or core.

The EHR-M standard provides a double disincentive. First, it puts the cost of certification on the innovator / developer. Secondly, it makes it uneconomical to divide modules into the very small components that independent developers can afford to build without financial backing.

There is no guarantee that a customized open source deployment would retain all of the features and functionality of the CCHIT-certified modules that it contains. For the purposes of maintaining quality standards EHR-M certification is meaningless.

Unfortunately, EHR-S certification won't fare much better. By using screen captures as a basis for EHR-S certification, CCHIT is committing to a process that's as worthless as using paper printouts for usability testing. An EHR that looks good on paper could be utterly unusable -- an expensive lesson that was learned bitterly in the 1990s by most dot-coms.

The choice of the word "site" in EHR-S is also quite unfortunate -- at times during the first Town Call, it was unclear as to whether CCHIT was using "site" to refer to a single location/organization or at a single web domain.

Alex I said...

I am in the process of having my Emergency Department software product
certified by the CCHIT. Understandably I'm chasing a moving target with whatever 2009-10 criteria happen to be by year end. I may be in the EHR-C or M category depending on final determination by the CCHIT. My biggest concern is that is I fail certification initially I must pay the entire certification fee again to be entirely retested (if I modify my software). If I don't modify the software then just the failed portion is retested. Given that CCHIT does not work like the SEI/CMM certification where there are Assessors that can be hired to do trial runs and help you mitigate your gap, the CCHIT certification can be very expensive and risky. No matter how many resources you place on the certification project you never know if an assumption you are making is right or wrong. My CCHIT contact has been fantastic but you'll never know until Certification Testing Day if you really have it right? I'm rambling, but my point is that CCHIT should allow a retest of just the failed portion regardless of software mods, at a nomimal fee, not $35,000 for a second time. And I do understand regression issues but I still think a concession is in order.

mxganse said...

"The cost to the vendor for certification is $30,000-50,000 per product."

Ouch. I guess everyone has to get a piece of the pie...or at least 30k worth of fingers in it.

Donald Green MD said...

The alternative, which may not be an unreasonable action, is to inform potential users how to evaluate software to match their needs and work with developers to bring about products that enhance practice. This may eliminate the certification process entirely. Physicians in many settings seems capable of putting other complicated procedures in place. Making an EMR work for them should be possible also.