I was recently asked by my staff how we should coordinate the time of day across organizations which exchange healthcare information. In a future which treats data from outside data sources as appropriate for clinical decisionmaking, you can imagine the following data exchange:
Hospital 1 posts lab result 12:01pm
Hospital 1 sends result to Hospital 2 12:02pm
Hospital 1 revises lab result 12:15pm
Hospital 1 sends revision to Hospital 2 12:16pm
Order is entered at hospital 2 12:17pm
Time synchronization among participants in a healthcare information exchange is important. If Hospital 2's clocks were 3 minutes slow, it would be challenging to know if the order was entered based on the original or revised lab result.
HITSP has published T16, the Consistent Time Transaction to help address this problem. It's based on an IHE profile created to support the synchronization of security audit logs.
Here is the relevant section IHE ITI TF Vol 2: 3.1.4.1 from IHE-CT profile
"The NTP transactions are described in detail in RFC1305. There is also extensive documentation on the transactions and recommendations on configurations and setup provided at http://www.ntp.org. Rather than reproduce all of that material as part of this Framework, readers are strongly encouraged to explore that site. The most common mode is the query-response mode that is described below. For other forms, see RFC1305 and the material on http://www.ntp.org.
The Time Server shall support NTP (which implicitly means that SNTP clients are also supported). Secure NTP may also be supported. The Time Client shall utilize NTP when it is grouped with a Time Server, or when high accuracy is required. For ungrouped Time Clients with 1 second accuracy requirements, SNTP may be useable. Time Clients may also support Secure NTP."
Although original designed for audit trails, the transaction has been expanded to other transactions, since organizations have realized that having synchronized clocks really helps documentation integrity and workflows. As the use of the consistent time is extended beyond audit trails, there are interesting issues about just how precisely synchronized devices in a network should be - a few seconds, one second, a subsecond?
At BIDMC, we point to stratum 1 servers that are directly connected to computers attached to atomic clocks.
The interesting question for HIEs is what should be synchronized.
My hospital servers are all synchronized against one set of time sources.
Our HIE, NEHEN, has suggested that all the gateways used to exchange data among multiple hospitals should be synchronized with one time source to ensure that all send and receive timestamps for clinical data exchange are consistent. Otherwise, data might arrive at one hospital before it leaves another!
However, since HIE gateways will be synchronized with one time source and hospital internal servers may be synchronized with others, the HIE time may vary from the hospital time.
Maybe the right answer is that as part of our national healthcare IT effort, we should mandate that all hospitals and HIEs should use a single set of known-adequate time servers to ensure all healthcare time is consistent.
For the moment, the following strategy seems reasonable
1. Require hospitals use NTP to ensure their internal time stamps are consistent. This will ensure audit trails within an organization, whether merged in an audit repository or just reported from disparate systems upon request, are consistent.
2. Synchronize health information exchange gateways within an HIE to a single time source so that transactions have consistent send and receive times.
If we know the hospital audit trail time stamp is consistent and we know the HIE send/receive times are consistent, we can recreate any event that is disputed.
Expecting every hospital to change its time synchronization servers to those used by the HIE is unrealistic - what if the hospital participates in multiple HIEs?
At some future time, we all may change to a national healthcare time server that is part of the NHIN, but for now the hospital use of NTP will be decoupled from the HIE use of NTP.
I think requirement §1 should be enough; if the relevant systems are running with NTP synchronization (either against internal time sources (GPS, radio), external (pool.ntp.org) or their communication partners), the drifts should be tiny fractions of a second. If you need higher granularity than this, I think real transactional systems are the only way to go.
ReplyDeleteA better §2 would be to ask that the providers to monitor the time drifts of their communication partners. This would likely detect large drifts, and there would be degree of dual control.
Actually, Concistent Time, as it was created by IHE, was not just intended for audit logging. There are a number of reasons why time synchronization is important in healthcare, not just audit trails.
ReplyDeleteIt happens to be very easy to implement on any platform. See http://wiki.ihe.net/index.php?title=Consistent_Time
I agree that audit is not the only, or perhaps even the most important, business justification for time consistency. Consistent time is equally important for system stability and service availability.
ReplyDeleteAlso, in addition to the NTP protocol, we may need standards around time source and stratum level, since each stratum level introduces further delay.
I agree that timing is critical. I do not think that requiring HIEs and participants to install timing systems or sync to other highly accurate time sources before exchanging data is unreasonable. A rubidium clock with PC interface, GPS based clock, or CDMA based clock are all readily available and very inexpensive in this day and age. This type of timing structure is commonplace in other industries to insure the proper operation of systems and accuracy of data. For healthcare environments we’ve found that having a single time source is valuable for things other than audit logs as well. Even simple time and attendance systems have benefited greatly and reduced complaints by staff when the PC clock does not match their timecard clock which doesn’t match the time display on phones for example.
ReplyDeleteIt might not be unreasonable for the HIE to query each connected organization daily to get a snapshot of their current timestamp and record it compared to the HIE's timestamp. This may help at a future date when trying to reassemble audit trails that span multiple organizations.
ReplyDeleteThis is an issue for organizations that span time zones. I have also seen issues with "legacy" hardware that does not sync to any time server...the process for setting the time on a mainframe may be "I look at my watch during the quarterly IPL"
ReplyDeleteWe cannot forget how much healthcare depends on legacy systems.
The US Military, aviation and several other industries have a simple solution for time synchronization, and it's called Zulu time or GMT (Greenwich Mean Time)(http://wwp.greenwichmeantime.com/info/timezone.htm). Everywhere on the planet Zulu time is the same. The local time zone is calculated in relation to the number of hours it is ahead or behind Zulu time.
ReplyDelete- For example, Eastern Standard Time is Zulu time - 5 hours.
- Central Standard Time is -6 hours from Zulu time.
- Japan is Zulu time +9 hours.
- Daylight savings time changes do not change Zulu time.
Using Zulu time (or GMT) to record event timing synchronizes events everywhere on the planet. Every time you travel by airplane (anywhere in the world) the flight uses Zulu time to record and coordinate all events related to the flight.
"Ditto" regarding the earlier time zone comment.
ReplyDeleteI'm personally a fan of using Greenwich Mean Time (GMT) when storing date/time in a database. It tends to isolate the chaos associated with moving around the date of Daylight Saving Time observance. I actually desire that time zones/DST be abolished to remove the possibility of off by 1 hour errors and help facilitate meet ups.
Minor nitpicking: The terms "Zulu time" and GMT has been superceded by Coordinated Universal Time (UTC): http://en.wikipedia.org/wiki/Coordinated_Universal_Time for this use. All system clocks should run on UTC, while time zone calculations will give a correct local time.
ReplyDeleteThe time coordination issue inevitably involves the measurement and retention of time offset(s), whether fractions of a second for time creep or hours for timezones. Time translation must occur somewhere in the 'federated' system. For the HIE, a natural choice would be UTC/UT1/Zulu. Each participating system would then perform time translation for it's local time zone only, and would not need to maintain multiple offsets for every constituent in the HIE federation. Legacy or otherwise, this ability is a requirement whether for HIEs across time zones (hours) or across town (seconds).
ReplyDelete