Cool Technology of the Week

When I was 18 years old, Sybex published my first book, "The Best of CP/M Software". That was followed by "Real World Unix", a roadmap to implementing Unix servers in small to medium size businesses, and "Espionage in the Silicon Valley", a collection of true stories about my experience working in high tech in the early 1980's.

Each of these was published in the traditional way. I submitted proposals to publishers, received a small advance, wrote manically for months, reviewed galleys, and then thousands of first editions rolled off the presses. The process was time consuming, expensive, and involved many stakeholders.

The Cool Technology of the Week is LULU.COM, which completely revolutionizes the publishing process by supporting on demand publishing of any author at low cost. At CareGroup, we recently used Lulu to publish a book about the retirement of one of our longstanding employees.

Here's how it works:
1. Publish -You upload your manuscript and photos, then use their online formatting tools to specify the layout, size and binding. There are no set up fees - you just pay the cost of the book when you buy it, as you would in a bookstore. Lulu does not inventory any books, they are digitally produced on demand when ordered.

2. Sell - Lulu books are listed in the Lulu marketplace, but you can also get an ISBN number so the book will be available on Amazon.com, in retail stores, libraries and schools.

3. Connect - Lulu includes social networking tools to link together a community of digital media creators.

When I compare the process of on demand book publishing to the workflow I used 30 years ago, I'm amazed at how democratic publishing has become. Like blogging, anyone can become a published author with formal publisher. In fact, if I wanted to bind together my first 100 blog posts in book form, I could do so without a set up fee on Lulu, call the result something like "True Confessions of a CIO" and make it available on Amazon. This is my 80th blog post, maybe I'll do that after my 100th.

Medication Reconciliation

One of the most challenging Joint Commission requirements for hospitals is supporting medication reconciliation workflow. This means that at every transition of care, providers must verify the medications that each patient is taking to ensure they are getting just the right dose of just the right medication. Why is this a challenge? Imagine that an 87 year old taking 14 cardiac medications visits an orthopedist for knee pain. The orthopedist must carefully record her current doses of Amiodarone, Lasix, Lipitor, and Zestril, examine her knee, prescribe medications, and document the visit, all within 12 minutes. For specialists, this may require a knowledge of medications they do not often prescribe and may take substantial effort if the patient has been referred from an outside provider and thus no medication history is available in the orthopedist's electronic health record.

To date, most hospitals and clinician offices have addressed this Joint Commission requirement by implementing paper processes. Very few vendors offer electronic solutions for medication reconciliation.

In July of 2007, BIDMC was visited by Joint Commission. We had been working on medication reconciliation workflow and software solutions for a year prior to this visit. During their visit, we were live with automated outpatient medication reconciliation, paper-based emergency department medication reconciliation, and paper-based inpatient medication reconciliation. The Joint Commission visitors were impressed with our software engineering but noted that we did not have 100% utilization of the software in our specialty clinics. We had 45 days to ensure 100% compliance, verified with an audit.

Over those 45 days, the Medical Executive Committee modified our clinical documentation policies to require medication reconciliation as part of retaining staff privileges. We assembled a multi-disciplinary committee to understand the workflow implications of medication reconciliation. We cleaned up historical medications by deleting all medications that had not been acted upon in three years. We temporarily hired 5 extra RNs to call patients at home and enter medication lists prior to their visits. When not calling patients, these RNs, who were located in our busiest clinics, cleaned up historical medication lists to ensure they were formatted properly for e-Prescribing renewals.

Our committee recommended hundreds of software changes to make the outpatient reconciliation process easier and we implemented all of them. Enhancements included the ability for any clinician in any clinic to quickly enter, update, or annotate (“patient is not taking as prescribed”) any medication, even if they were not the original prescriber. Of course all medication changes were documented in an audit trail. We enabled providers to enter patient self reported medications and we displayed an alert if the exact dose was unknown.

We modified our eprescribing systems to reflect community-wide medication history. This means that when a clinician writes for any medication, a history of every medication that has been dispensed at any US pharmacy appears and can be used as an aid to reconciliation.

We modified our emergency department clinical documentation system to support reconciliation of patient medications during ED evaluations and ensured all medications dispensed in the ED became part of the patient's active medication list.

On the inpatient side, we enabled discharge medications to be automatically converted to outpatient medications and made all discharge medication summaries available to every clinician. We are just finishing a completely automated inpatient medication reconciliation system which will enable physicians to verify all medications electronically while preparing the initial inpatient history and physical.

Our paper-based medication reconciliation processes will soon be retired.

The greatest challenge to implementing all these solutions was documenting the logical workflow required by clinicians, then getting the buy in of all the users of the system, given that it added substantial work to their day.

At this point, 5 months after our final joint commission audit, physicians are realizing the benefit of an up to date medication list and are saving time in the medication renewal process, receiving more accurate decision support when prescribing new medications, and have better documentation of patient compliance with their therapies. Patients are more satisfied as well, since they know all clinicians will use the same accurate medication list across our hospital and practices, making patient/provider communications about medications much easier.

It may have been one of our most challenging experiences, but in the end, all involved agree it was worth the pain.

Measuring Programmer Productivity

In my role as CIO at CareGroup and Harvard Medical school, I oversee nearly 100 software developers. Many organizations purchase software or outsource development to avoid managing developers, but we Build AND Buy and will continue to do so. I was recently asked to share the metrics we use to evaluate our developers. Here is our framework:

1. How may applications does the developer maintain? Applications can vary in size and complexity, so that must also be documented. This helps us understand how many staff are needed as our application portfolio expands.

2. How many new applications can a developer create annually? Some programmers develop version 1.0 of new applications, while others maintain existing applications or modules. This helps us understand our development skill set and our ability to take on new development projects.

3. What is the lifecycle stage of each application assigned to a developer? (New, mature, need to replace within 2 years). This is a reasonable way to measure work completed and forecast upcoming work.

4. How much bug free code is developed per year? This is an imperfect measure, since it measures quantity, not quality, but it is useful to understand as a proxy for coding productivity.

5. Are the application stakeholders satisfied with the quantity and quality of application features developed?

We piloted these measures at Harvard Medical School by

1. Inventorying all the major web applications by developer, including the technologies used.

2. Categorizing this inventory into applications which are new and those which are in maintenance. We also rated each application for intensity of support (High, Medium, Low) based on the frequency of code change that has been historically required to maintain it

3. Rating each application's lifecycle stage.

4. Documenting the number of application releases by each developer over the past year.

5. Creating a survey to measure user satisfaction with each application.

In the end, we published a detailed scorecard for each developer and a summary for all developers. This data alone doesn't tell the whole story, but it does help us plan and better manage our development staff.

Data Center Costs

One of the great challenges of being a CIO is that I am asked to anticipate demand and deploy the infrastructure and applications needed by stakeholders, while tightly controlling costs. This means that I am rewarded for meeting demands just in time, but penalized for under provisioning (not meeting demand) or over provisioning (spending too much).

Even in a world of virtualization and on demand computing, many IT products and services have a significant lead time to deploy. Creating a new data center, adding significant new power feeds and expanding cooling capacity are examples. To help understand the relationship of real estate, power, cooling, storage, network bandwidth and costs, we've created a web-based model which we're now sharing with the IT community. Feel free to access it here.

You'll see that this dashboard enables you to change costs per square foot, costs per kilowatt, total server/storage capacity and other variables to create a customized cost dashboard. At Harvard Medical School, we've used this dashboard during the budget process to achieve predictable budgeting. CFOs do not like surprises, so agreeing on a predictable cost model that directly relates demand, supply and cost, makes life for the CIO much easier.

I hope you find this useful!

Electronic Records for Non-Owned Doctors

In my previous post The top 10 things that will keep me up at night in 2008, the number one project is providing electronic health records (EHR) to non-owned physicians.

This will be the first of a series of posts about this very complex project. My posts will detail the cost challenges, the partnership we've built to execute the project, and the technical approach we've taken. Since this is truly a work in process, I'll publish these posts each week over the next few months.

Since 2002, my IT teams have provided electronic health records to every owned/closely affiliated clinician of Beth Israel Deaconess, using our home built webOMR software. We even have a Medical Executive Committee policy mandating the use of electronic health records for owned clinicians by July 30, 2008. 'Use' is carefully defined, since we want all our clinicians to update the problem list, create an electronic note for each encounter, and perform all medication management (e-Prescribing, medication reconciliation, allergy documentation) electronically. Of interest, recent data collected by David Blumenthal of Massachusetts General Hospital concludes that only 4% of clinicians in the US have this level of use of fully functional electronic health records.

Since Stark safe harbors now enable hospitals to fund up to 85% of the non-hardware implementation costs of private practice electronic health records, my teams are now expected to provide EHR solutions for all the non-owned affiliated BIDMC affiliated physicians in Eastern Massachusetts. This is a very different project than providing applications and infrastructure to owned clinicians, which we manage, on a network, which we manage.

The planning for the project includes the following major issues:

1. Designing Governance – My typical steering committees are drawn from hospital senior management, employed clinicians and hospital staff. The governance of a community-wide electronic health record system must include members of the physicians' organization, private physicians, community hospital executives, and legal experts.

2. Modeling costs – The 'hydraulics' of the project budget are quite complex. The hospital wants to support implementation for as many physicians as possible but has limited capital. The physicians want as much implementation subsidy as possible since by Stark regulation they have to fund all hardware and ongoing support themselves. The number of doctors implemented, the level of subsidy and total costs for all stakeholders are interrelated but each party has different goals.

3. Planning for distributed users – These non-owned clinicians are widely dispersed throughout Massachusetts and New England in urban and rural settings. Bandwidth varies from 20 Megabit Verizon FiOS connections to 56K dialup. Technology sophistication varies from fully 'wired' clinicians to offices run on 3x5 index cards.

4. Managing the project – CIOs traditionally serve hospital-based customers. They may not have the bandwidth or expertise to serve non-owned geographically dispersed customers.

5. Building a scalable infrastructure - The architecture must be designed to minimize costs, maximize reliability and support a project scope that is continually evolving.

6. Deploying staff - The existing hospital IT staff is not optimized for supporting networks, telecom, desktop and application at hundreds of remote locations. The physicians' organization and clinician offices do not have the staff or expertise to execute this project.

7. Creating the Model Office – A clinically integrated network of providers in a community will want to adopt a standard EHR configuration with common dictionaries to support healthcare information exchange and continuity of care. Standardizing software configuration means standardizing workflow, which requires business process re-engineering. Practice consulting is needed to balance standardization with specialty specific processes, ensuring that providers buy into the new workflow and staff are appropriately trained.

8. Obtaining all the funding - Once the scope, architecture, staffing and cost modeling is completed, the funding must be obtained from all of the stakeholders. State and Federal governments are not likely to contributing anything. Payers may fund the outcomes of EHR use via pay for performance, but are unlikely to pay for implementation.

9. Specifying the order of implementation - How do we choose the most appropriate offices for pilots and once those pilots are completed, how do we place hundreds of clinicians in a well ordered queue for rollout?

10. Supporting the practices – Hospital support models depend upon standardized networks and desktops with end to end control over the quality of service. Supporting heterogeneous practices requires on site, high touch, higher cost service.

My goal is to create a blog entry for each of these issues. Next week, I'll publish the governance model. The following week, I'll post the detailed considerations we're using to develop the cost model (the finished models will be published under 'obtaining all the funding', since they are still evolving). By the time all 10 posts are done, we'll be live in pilots with 4 practices and I hope to be able to sleep again.

Cool Technology of the Week

In my previous posts, I've described our web applications of CareGroup and Harvard Medical School that run anywhere on anything. At some point, as more and more software is offered via Software as a service providers, Web 2.0 applications, and operating system neutral thin clients, the notion of a desktop becomes a basic operating system running a browser. Such a device can be a very thin client without much local storage.

We've investigated several thin client computers in the past, but the Cool Technology of the Week is the Jack PC from Chip PC Technologies. It's slightly larger than a pager fits inside a wall mounted network port. It's powered over ethernet, offers VMware support and connectivity to Microsoft Terminal or Citrix servers. Features include

• Installable in wall, furniture or floor
• Network integrated
  
• 100% theft-proof
• 100% virus-immune
• 100% data-secured
• 100% remotely managed
• Energy saver (3.5W)

Just plug a keyboard and flat panel screen into the wall and you're up and running! Very cool.

On the subject of small PCs, one followup from my post about the MacBook Air. In a recent article about the Air in AppleInsider, I found a fascinating comparison of major subnotebook manufacturers, that aligns with my own experience

Sony targets high end consumers; it leverages its physical media engineering prowess to build DVD burners into most of its models, something that few other light notebook makers even attempt to do. Sony's Vaio line is splashy and feature rich, but isn't commonly regarded as well built or durable.

Panasonic is known for its ruggedized Toughbook line, designed to operate in rough environments. Its models commonly trade off high end performance and features for extremely light weight and compact size. That relegates Panasonic's fans to mobile business users, and makes it less appealing to mainstream consumers.

Lenovo, which bought up IBM's PC division, continues the venerable ThinkPad line as a highly regarded workhorse that delivers top performance in a thin but well constructed case -- all work and no play. ThinkPads are also known for their long usable life and their fingertip controllers rather than trackpads, something that polarizes users for or against based on their personal preferences.

Fujitsu is another leader in light and thin notebooks, but also makes more general purpose machines that borrow from its leading edge thin designs. Its larger sized lines are powerful and economical while still remaining thin and fairly light. Fujitsu also makes Tablet PC convertible machines with the flip-around monitors that have yet to prove popular because they are expensive.

Dell was not discussed in the article, but my experience is that the Latitude line, such as the D420 with a wide screen, are light, durable, and economical.

At CareGroup and Harvard Medical School, we deploy Lenovo and Dell Latitude laptops.

Here's the complete article

Central and Local Information Technology

Recently, I was asked to write an overview about day to day IT support at Harvard Medical School, including ideas about expanding services over the next year. One of the most savvy folks in the research community responded with a great dialog that illustrates the balance between central and local IT. I've italicized his comments below:

1. Networks
Continue to provide high bandwidth wired and wireless networks to all HMS departments. Provide 100 Megabit connections to all desktops and 1 Gigabit connections as needed, assuming that appropriate Cat 6 wiring is available in the department. In parallel replace all legacy wiring at HMS, over the next 5 years to ensure all stakeholders have the potential for gigabyte connections.

This level of infrastructure is naturally a central IT (and physical infrastructure) function. However, the requirements are not entirely generic; e.g. different sites have specific network architecture needs. Support for the software side of networking (firewalls) with adequate attention to the particular needs of departments is also essential.

2. Servers
Continue to provide Unix/Linux and Windows server hosting with 24x7 management, operating system patching and security services for all HMS stakeholders. The offiste data center should be expanded to meet growing demand for server and high performance computing hosting.

This is one of the areas in which there is the most variation in the benefits of centralization to different research units. Some departments or labs have well-developed IT support staffs and server infrastructure, others have local servers but are poorly staffed, and some have little local support and are reliant on central services. For the first group, having local support staff responsible to a local group and dedicated to the particular needs of the site is a highly effective model. Whether this is consistent with centralized physical location of server hardware remains to be determined. For the other groups, some further investigation is needed to determine whether a centralized or dedicated server model is more appropriate.

3. Storage
Provide 50 Gigabytes per user and 500 Gigabytes per lab at no charge. Provide storage and archiving services to high volume users for $1/Gigabyte per year. To ensure that storage needs are met for all new faculty, include funding for IT infrastructure such as storage in their recruiting packages. Storage includes 24x7 support for high speed fiber channel or SATA Network Attached Storage, including appropriate means to attach to this storage for Mac OS X, Linux, Unix and Windows as well as web-based Online Storage. Archiving/backup is also provided as a service.

The proper organization of storage is closely linked to the organization of servers; the physical link between storage and server is still important. Backups might be efficiently provided as a school-level commodity service if competitively priced with an adequate service level for restores, although there are issues of confidentiality and data use agreements affecting some datasets.

4. Desktops
Continue to maintain help desk services, jointly managed with the departments, for support of Windows, Macs, and Linux machines. Provide anti-virus software and operating system updates.

Joint management with departments has been a good model. Some desktop-oriented applications and services are standardized commodities, but assignment of support staff to departments allows them to gain familiarity with the particular needs and configurations of departments and individual users.

5. Applications
Today, the application suite at HMS includes web portals such as Mycourses/eCommons; Administrative applications such as FIRST, Room Scheduling, MARS-Medical Area Reporting System, MAES- Medical Area Equipment System, Peoplesoft, GMAS, MADRIS; Research applications that run on Orchestra such as Twiki, Matlab, LaserGene, and Endnote; and Educational applications such as virtual microscopy; streaming video/podcasting; and simulations.

E-mail is another crucial service that is best supported at the school level due to the high cost of effectively dealing with spam and malware.

Expand this suite of applications through the enhancement of Mycourses/eCommons to include collaboration services such as instant messaging, Webex meetings, CONNECTS (a match making service for equipment, techniques, and scientists), SHRINE (a means of data mining across all Harvard affiliates), and web content management for easy hosting of internal and external websites. Expand this suite of applications to support research administrative and CTSA needs such as IACUC and animal ordering. Expand application support to include informatics services per the BRIC business plan such as database consulting and web application design.

Some attention should be given to the cost allocation model for these services since some of them are of general use, some are sometimes provided by internally supported staff at departments, and some are of use only to particular groups. However for the most part the items listed here are of general utility.

6. Disaster Recovery
Currently, HMS maintains two data centers and has significant redundancy in storage, server clusters, and networks. However, it does not have a true disaster recovery center and plan. Hire staff and develop a plan to ensure business continuity in the case of a major data center or natural disaster.

7. Media Services
Provide media services support for the entire school including presentation services, digital photography/videography, and teleconferencing services. The Media Services infrastructure is currently under review and school wide enhancements will be proposed in 2008.

As you can see from this dialog, neither completely central nor completely local is the right model for a research-focused medical school.

At HMS, central organizations provide "heat, power, light, networking and terabytes" - the utilities needed to empower all the core businesses of HMS - education, research and administration. Centralizing those utilities which can achieve economies of scale and reliability, while leaving local those areas of science and application expertise unique to each department, has worked well to support all our stakeholders. One of the major central/local collaborations has been the joint hiring customer service representatives for each department and locating them within the departments they serve. This enables each department to have centrally trained, managed, and budgeted staff but with specific skills to meet each department's academic needs. Similiarly, the HMS data center hosts centrally managed servers and storage but also hosts department specific infrastructure managed by scientists.

This division of labor between the IT department and the scientific community leverages the skills of each, ensuring a positive working relationship, based on a transparent governance model, for all the services we deliver.