What If We’re Wrong? Prehospital ECG Interpretation

This article is part of a special one-day EMS blogging event called the What If We’re Wrong a-Thon. Spearheaded by Brandon Oto over at EMS Basics, the WIWWAT is an exercise in self-reflection where EMS writers examine a topic on which that they’ve historically taken a strong stance, but from the opposite point-of-view. For more posts from around the EMS community as part of the What If We’re Wrong a-Thon, check out this page.

Click image for source.

Click image for source.

The Question

Despite years dedicated to cause of prehospital STEMI recognition and EMS interpretation of electrocardiograms, what if we’re wrong? What if there’s a better, safer, and more efficient method of triaging which patients need emergent PCI that doesn’t rely on the prehospital provider’s interpetation of the ECG? Perhaps the time and energy we spend in and out of the classroom training medics how to recognize ischemia could be better spent on other topics where we really can make a difference.

So the question we want to ask is “Should prehospital providers be identifying STEMI’s?”

The answer is “No.”

 

The Evidence

Bhalla MC, Mencl F, Gist MA, Wilber S, Zalewski J. Prehospital electrocardiographic computer identification of ST-segment elevation myocardial infarction. Prehosp Emerg Care. 2013 Apr-Jun;17(2):211-6. doi: 10.3109/10903127.2012.722176. Epub 2012 Oct 15. [PubMed]

These authors looked at 200 prehospital ECG’s from patients who were treated at their facility—100 from patients who were sent to the cath lab and 100 randomly selected controls who were not deemed to have experienced a STEMI. The golden standard was the emergency physician’s decision whether or not to activate the cath lab. The specificity of the ***ACUTE MI SUSPECTED*** message was found to be 100% (100/100; 95% CI 0.96-1.0) while its sensitivity was 58% (58/100; 95% CI 0.48-0.67).

  • Computers were 100% specific but they missed 42% of physician-activated STEMI’s in this study.

 

de Champlain F, Boothroyd LJ, Vadeboncoeur A, Huynh T, Nguyen V, Eisenberg MJ, Joseph L, Boivin JF, Segal E. Computerized interpretation of the prehospital electrocardiogram: predictive value for ST segment elevation myocardial infarction and impact on on-scene time. CJEM. 2014 Mar;16(2):94-105. [PubMed]

This study was performed in Canada. It looks at 1,247 prehospital ECG’s acquired by Primary Care Paramedics (PCP’s) over a 2-year period and examines the accuracy of the computerized interpretation provided by the GE-Marquette 12SL algorithm. The program was found to have a sensitivity of 69.2% (59.0-78.5) and a specificity of 98.9% (98.1-99.4).

  • The computer performed quite well in this study compared with some of the others. The sensitivity is still rather poor but the specificity is great.

 

Kudenchuk PJ, Ho MT, Weaver WD, Litwin PE, Martin JS, Eisenberg MS, Hallstrom AP, Cobb LA, Kennedy JW. Accuracy of computer-interpreted electrocardiography in selecting patients for thrombolytic therapy. MITI Project Investigators. J Am Coll Cardiol. 1991 Jun;17(7):1486-91. [PubMed]

Prehospital ECG’s were examine for 1189 patients with chest pain of suspected cardiac etiology. 52% (202/391) of patients diagnosed with AMI on discharge (not necessarily STEMI) were read as STEMI by the computer, compared with 66% (259/391) read as STEMI by the cardiologists. 98% (785/798) of patients diagnosed as not having experienced an MI were correctly excluded by the computer while 95% (757/798) of patients were excluded by the cardiologists.

  • Computers are again less sensitive but more specific than human interpreters. Also of general interest, most patients with suspected cardiac chest pain are not experiencing an acute MI.

 

Swan PY, Nighswonger B, Boswell GL, Stratton SJ. Factors Associated With False-Positive Emergency Medical Services Triage for Percutaneous Coronary Intervention. West J Emerg Med. 2009 Nov; 10(4): 208–212. [PubMed]

During the study period paramedics in Orange County, California triaged 548 patients for the cath lab. 393 (74.3%) were considered true-positives by the presence of coronary lesions treated with angioplasty or CABG while 136 (25.7%) were considered false-positives, either because the emergency physician cancelled the activation (121 patients; 22.9%) or because no culprit was found on cath (15 patients; 2.8%). The key to this study is that the act of the paramedics activating the cath lab was determined solely by the computer generating an ***ACUTE MI SUSPECTED*** message. For more on the particulars seem Tom Bouthillet’s analysis.

  • This paper was surprising for the number of false-positives since computerized algorithms in general seem to have a pretty good specificity. The exception here might be that one particular brand fell out as being responsible for a preponderance of the false-positive activations.

 

Mencl F, Wilber S, Frey J, Zalewski J, Maiers JF, Bhalla MC. Paramedic ability to recognize ST-segment elevation myocardial infarction on prehospital electrocardiograms. Prehosp Emerg Care. 2013 Apr-Jun;17(2):203-10. doi: 10.3109/10903127.2012.755585. Epub 2013 Feb 12. [PubMed]

Paramedics were administered a survey with 10 ECG’s: 3 STEMI’s, 2 normal tracings, and 5 STEMI mimics. 472 medics responded with an overall sensitivity of 75% (95% CI 73%-77%) and specificity of 53% (95% CI 51%-55%) for the recognition of STEMI.

  • Paramedics in this study were 75% sensitive but only 53% specific for the detection of STEMI. That’s terrible.

 

Cantor WJ, Hoogeveen P, Robert A, Elliott K, Goldman LE, Sanderson E, Plante S, Prabhakar M, Miner S. Prehospital diagnosis and triage of ST-elevation myocardial infarction by paramedics without advanced care training. Am Heart J. 2012 Aug;164(2):201-6. doi: 10.1016/j.ahj.2012.05.009. [PubMed]

This study was designed to assess the ability of Primary Care Paramedics (a designation unique to Canada) to recognize and manage suspected STEMI. 134 patients were triaged for urgent PCI by the medics involved. 106 cases were deemed to be true STEMI’s in follow-up. The computerized interpretation recognized 98 of the 106 true STEMI’s but was fooled by 17 of the 28 false-positives. There was agreement between the prehospital provider’s interpretation and that of the physician 90% of the time.

  • The computer showed excellent detection of STEMI in this case but poor ability to discern mimics. This is a bit of an outlier compared with other ingestigations of computerized algorithms. The medics performed well in this study but their experience and training is difficult to compare with that of the U.S. EMS system.

 

Kontos MC, Kurz MC, Roberts CS, Joyner SE, Kreisa L, Ornato JP, Vetrovec GW. An evaluation of the accuracy of emergency physician activation of the cardiac catheterization laboratory for patients with suspected ST-segment elevation myocardial infarction. Ann Emerg Med. 2010 May;55(5):423-30. doi: 10.1016/j.annemergmed.2009.08.011. Epub 2009 Sep 20. [PubMed]

This study looked at 249 cath lab activations initiated by emergency physicians at their center. 188 patients (76%) were diagnosed as experiencing a true STEMI but only 5.2% of activations were deemed “inappropriate” by cardiology.

  • Be careful how you define “false-positive” and “appropriate” with regards to STEMI activation. I think that’s all we can say about this one.

 

O’Donnell D, Mancera M, Savory E, Christopher S, Schaffer J, Roumpf S. The availability of prior ECGs improves paramedic accuracy in recognizing ST-segment elevation myocardial infarction. J Electrocardiol. 2015 Jan-Feb;48(1):93-8. doi: 10.1016/j.jelectrocard.2014.09.003. Epub 2014 Sep 16. [PubMed]

130 paramedics were tested for their ability to recognize STEMI on 12 randomly selected prehospital ECG’s; 6 with an accompanying baseline ECG and 6 without. The addition of old ECG’s improved the paramedics’ accuracy from from 75.5% to 80.5%.

  • Paramedics were 75% accurate in recognizing STEMI, but that could be improved with the addition of old ECG’s (as are available in-hospital).

 

Whitbread M, Leah V, Bell T, Coats TJ. Recognition of ST elevation by paramedics. Emerg Med J. 2002 Jan;19(1):66-7. [PubMed]

7 London paramedics were tested in their ability to recognize “ST-elevation” (not necessarily STEMI) in 100 ECG’s as compared with the gold-standard to two cardiologists. The medics were 97% sensitive, [CI 0.94–0.99] 91% specific [CI 0.53–1.0], and 95% accurate [CI 0.88–0.98].

  • It seems these medics did quite well but quoting from the paper’s discussion: “This study was performed using a group of well motivated and enthusiastic paramedics from a single ambulance station with a close connection to a teaching hospital. The same results might not be obtained if all paramedics were trained in this skill.” Also, they were not specifically tested for their ability to recognize STEMI but rather a few rather specific patterns of ST-elevation they had been taught in a class 12-months prior. It pays to read the full paper and I think these numbers are grossly inflated from what they would be in the real world.

 

Trivedi K, Schuur JD, Cone DC. Can paramedics read ST-segment elevation myocardial infarction on prehospital 12-lead electrocardiograms? Prehosp Emerg Care. 2009 Apr-Jun;13(2):207-14. doi: 10.1080/10903120802706153. [PubMed]

103 Connecticut paramedics were given five scenarios that tested their ability to recognize the three ECG’s that showed true STEMI and two that did not warrant cath lab activation. Overall, the paramedics were 88.0% sensitive (83.8–91.3) and 88.3% specific (83.0–92.2)  in activating the cath lab with an 8.1% (5.4–12.0) false positive rate.

  • These paramedics performed well identifying a very limited number of obvious STEMI’s. Because they were computer-generated ECG’s it’s reasonable to assume the computerized algorithm would have been 100% sensitive and specific in identifying these STEMI’s.

 

Feldman JA, Brinsfield K, Bernard S, White D, Maciejko T. Real-time paramedic compared with blinded physician identification of ST-segment elevation myocardial infarction: results of an observational study. Am J Emerg Med. 2005 Jul;23(4):443-8. [PubMed]

151 prehospital 12-leads (25 of which were judged as showing STEMI by a reviewer using standard millimeter criteria) were examined by blinded emergency physician and a blinded cardiologist. Their interpretations were compared with the initial interpretations performed by the paramedics in the field. The paramedics showed a sensitivity for STEMI of 0.80 (95% CI, 0.64–0.96) and a specificity of 0.97 (95% CI, 0.94–1.00). The overall accuracy was similar for the paramedics, emergency physician, and cardiologist (0.94, 0.93, 0.95; respectively).

  • Paramedics performed well in this study but it must be noted that they constituted a “stable force of highly trained, motivated, and experienced ALS providers” an urban EMS system and would be expected to perform better than the national average. Also, it’s important to note that there was an inordinately high rate of STEMI in the population studied, as mentioned below…

01 - Feldman et al.

 

Le May MR, Dionne R, Maloney J, Trickett J, Watpool I, Ruest M, Stiell I, Ryan S, Davies RF. Diagnostic performance and potential clinical impact of advanced care paramedic interpretation of ST-segment elevation myocardial infarction in the field. CJEM. 2006 Nov;8(6):401-7. [PubMed]

Another Canadian study, this time examining Advanced Care Paramedics (ACP’s). According to the reviewers (two emergency physicians and one cardiologist), during the study period the ACP’s correctly identified 63 STEMI’s, missed 3 cases, and had 13 false-positives. Their sensitivity was 95% (86%-99%) with a specificity of 96% (94%-98%).

  • Canadian training is very different from the American system and that needs to be kept in mind. Also, with only 3 “missed” STEMI’s reported, I suspect there were a lot more not picked up by the reviewers…

 

Davis DP, Graydon C, Stein R, Wilson S, Buesch B, Berthiaume S, Lee DM, Rivas J, Vilke GM, Leahy DR. The positive predictive value of paramedic versus emergency physician interpretation of the prehospital 12-lead electrocardiogram. Prehosp Emerg Care. 2007 Oct-Dec;11(4):399-402. [PubMed]

This study was performed in two phases. In Phase I paramedics activated the cath lab on their own and in Phase II ECG’s were transmitted to emergency physicians who then activated the cath lab. 110 patients were enrolled (54 in Phase I, 56 in Phase II). In Phase I cardiologists confirmed STEMI on the prehospital ECG in 78% of cases (42/54), emergent PCI occurred in 70% (38/54), and culprit lesions were found (or the patient arrested) in 69% (41/54). In Phase II cardiologists confirmed STEMI in 96% of cases, emergent PCI occurred in 91% of cases, and culprit lesions were found or the patient arrested in in 89%.

  • Allowing the emergency physician to activate the cath lab via transmission significantly improved the accuracy of the decision.

 

Garvey JL, Monk L, Granger CB, Studnek JR, Roettig ML, Corbett CC, Jollis JG. Rates of cardiac catheterization cancelation for ST-segment elevation myocardial infarction after activation by emergency medical services or emergency physicians: results from the North Carolina Catheterization Laboratory Activation Registry. Circulation. 2012 Jan 17;125(2):308-13. doi: 10.1161/CIRCULATIONAHA.110.007039. Epub 2011 Dec 6. [PubMed]

14 primary PCI hospitals were examined over 12 months, during which time there were 3973 cath lab activations. 29% were performed by prehospital providers were 71% were called by emergency physicians. 596 cases were deemed “inappropriate” activations, with EMS responsible for 242 of them (40.6%) due to incorrect ECG interpretation while emergency physicians were only responsible for 183 cases (30.7%) despite activating the cath lab more times.

  • Yet again emergency physicians demonstrate their superiority in regards to correct activation of the cath lab.

 

Some Conclusions

After sorting through that huge pile of information, let’s start drawing some conclusions. Here’s the big three I’ve gathered from the above studies.

  • Computers are specific but not very sensitive when it comes to recognizing STEMI. When they say it’s a STEMI it usually is, but they also miss a lot of them. Their performance can be improved by ensuring good-quality tracings with mimimal artifact and no “Data quality precludes interpretation” message, but there is still a ceiling to their abilities.
  • Prehospital providers are more sensitive but often less specific than computerized algorithms. Also, their performance varies greatly from region to region.
  • Emergency physicians are at least as good as prehospital providers (at least in studies) and often significantly more accurate at ECG interpretation. They also initiate less false-positive and cancelled activations.

So, if you want optimal numbers, your system should be transmitting 100% of its ECG’s to an emergency physician base station for interpretation and activation. I hope that answers the question we set out to address, and true to our goal from the start, it’s certainly a sobering pill for me, your ECG-obsessed author.

 

 

…there are some other considerations to keep in mind though.

  • It is expensive to run that sort of setup unless you work in a large system with an academic center willing to handle the influx of ECG’s.
  • ECG’s cannot be transmitted from everywhere that there is radio contact, so there are times when a medic would be able to activate well before getting into transmission range.
  • There are systems with highly-trained, well-seasoned, eager medics capable of reading ECG’s as well as emergency physicians. In addition to saving money, it also encourages excellence amongst the providers because they know they are the ones who will be making important decisions and not just pressing a button for someone else to do the work.
  • The person in the field is also the person who can see the patient, and quite often that plays a big role in whether an ECG is concerning or not. It is much more difficult to perform a cold-read on an ECG than it can be with the patient in front of you.

There are certainly many more variables that go into deciding what setup will work best for a particular region, but that’s a start. Admittedly, though, those are all exceptions when the rule is that, taken in toto, emergency physicians are a better bet when it comes to accurate STEMI activation.

What I think is clear is that there are two major possibilities in play: physician interpretation and medic interpretation (with the computerized interpretation turned on please); along with combinations there-of. And what I think is clearly inferior to those options is a system where cath lab activation is based solely on the computer’s message of:

***ACUTE MI SUSPECTED***

It’s nice when it’s there, but like most folks who type in all caps, the computer can be oblivious to some things that are pretty obvious to the rest of us.

 

For more self-contradictory posts from the What If We’re Wrong a-Thon, check out this page.

3 Comments

  • Jake says:

    One thing that seems to stand out is that EMS systems with good 12-lead training seemed to perform on a level approaching or equivalent to ED physicians.

    My impression, from what I see in these blogs and discussion groups, as well as my own personal experience, is that 12-lead acquisition and interpretation is being poorly taught in many – possibly most – US EMS systems. I know my own “training” on the subject, when my agency first got 12-lead capability, was simply pitiful (basically, a 1 hour “here’s how you do it” CE session, that seemed to assume everyone there had already been taught basic interpretation). I’m much better now, but what I’ve learned has all been from seeking out classes on my own, or from sites like this (Thank you!).

    If this education deficiency can be corrected, then we should be able to get EMS cath lab activations up to the same specificity/sensitivity rates as ER physicians, and make the proper answer to this question “yes”.

  • Erik says:

    In my paramedic class we were taught to pay attention to the numbers generated by the monitor, but you can never trust them. You must examine the rhythms in each lead and look for appropriate elevations along with reciprocal depressions. And this is not a time consuming event. I will say that in my city we do have the ability to transmit all ECGs via LifeNet on our LifePak15’s. We can send only to the ED, but if we suspect a STEMI or other cardiac event we can send to the ED, cardiology, medical director, and EMS director all at once. So we do always have that safety net.

  • Ambitious but EXCELLENT post by Vince. I think this is a “moving target” with ongoing and evolving “Conclusions” to match the preliminary Conclusions suggested at the end of the article by Vince — with the important NOTE that what works in any given part of the country (or world, for that matter) is highly specific to a variety of factors including set-up of EMS access and Emergency Care + Cath Lab availability — ED and EMS expertise (and enthusiasm) for acute ECG interpretation — and ECG transmission capabilities. My point being simply that while Vince’s Conclusions are an excellent start — each EMS system needs to examine these questions based on their own unique composite situation. I favor ED transmission of ECGs in most cases, not because EMS personnel can’t read them — but rather my belief that optimal decision-making is often a joint effort (in my opinion) with not only the ED physician, but also often the Cardiologist being involved from the start. There are shades of “gray” that in my experience may be assessed differently depending on which of a number of highly-capable cardiologists in a given coverage team happens to be On Call — so nothing beats that cardiologist On Call seeing the initial ECG via transmission at an early point in the system to aid in collaborative decision-making.

    As to transmission capabilities — at least in the U.S. — cell phone transmission should be possible almost anywhere within no time at all. I realize that there are medicolegal concerns on certain aspects of this — but given primary priority being best care of the patient, I still fail to understand why cell phone transmission isn’t used more often when other transmission means are (for whatever reason) delayed. Cell phone transmission takes literally no more than seconds — and a reasonable ECG picture is then sent within seconds to ED staff (and if/as desirable, to the cardiologist On Cal who may need to come in to do the acute cath).

    Finally — I submit that the optimal approach is a combination of all factors Vince mentions. That is, under certain circumstances ED staff should become totally comfortable with EMS expertise in assessing the “definite cath lab activation” cases, even without need to review the ECG (if transmission is for whatever reason delayed) — because after time, ED staff becomes very comfortable with interpretation abilities of specific EMS providers they have worked with and know. By the same token — this implies need for recognition on the part of EMS staff about which cases/tracings there should be questions (“gray areas”), and for which transmission to involve ED staff (if not also the Cardiologist On Call) is clearly the BEST policy.

    BOTTOM LINE: This process continues to be a work in evolution. I have been totally impressed by the expertise shown by MANY EMS providers in acute ECG interpretation — such that (in my opinion) regardless of study results that Vince reports suggesting superiority of ED staff assessments — it is not an “all or none” phenomenon, but rather a complex process for which a basic structure with potential flexibility for individual system specifics would be the approach I’d favor … Collaboration between EMS + ED physicians + cardiologist On Call is KEY to optimal patient care.

1 Trackback

Leave a Reply

Your email address will not be published. Required fields are marked *