The confusion matrix part II: the probability of the posterior depends on the sensitivity, your crazy friends that ruins everything, and would you like a pamphlet?

This is part two in a three part series.

Part one of this three part series asked a few questions about what the probabilities are that a patient with chest pain is having a myocardial infarction and the answers are below in this blog.

Lots of people seek emergent medical treatment for chest pain. Most chest pain ends up being something other than ACS; things like pneumonia, anxiety, costochondritis, referred pain from an organ, a pulmonary embolism – the differential is lengthy and everyone knows it.

The first question in the previous post asks about how likely it is that the patient is having ACS based only on the dispatch information.

This first question is all about prevalence – how many people in a given population have this condition. The odds that this is ACS are fairly low just based on the dispatch information. The best data on this comes from patients with chest pain seeking medical treatment at an ED that are eventually diagnosed with ACS. Between 13% and 14% of patients seeking emergent treatment for chest pain are eventually diagnosed with ACS.

question 1 draw
Question 1 was: With only the dispatch information provided, what do you think the chances are that this patient is having a myocardial infarction? 

The way to read this chart might be a bit confusing at first. If you look at the first column, 21.8% of the people who answered this question felt there was less than a 10% chance this was an infarct based just on the dispatch info. The blue column shows that 28.2% of people who answered the question thought that the chances were between 11-30% that this was ACS based just on dispatch info.

After getting on scene in the scenario and seeing the patient a lot people changed their answers to a higher probability that this is ACS.

sketch1570244644166
Question 2: With only the dispatch information provided, what do you think the chances are that this patient is having a myocardial infarction? This is the pre-test probability. At this point no one thought the chances were less than 10% that this was an infarct.

In light of new evidence (typical chest pain, arm pain, gestalt, etc.) we should update our views of the probability. and this goes both ways—if the patient was busy playing angry birds and looked up at you and smiled while they said their pain was a 10 out of 10 you might update your estimates of probabilities in the opposite direction.

In a few cases this can be accomplished using tools or this can be based on an educated guess if there aren’t validated metrics out there. I fathomed a guess of around a 50% probability and I think it was pretty close based on this tool. Pre-test probability calculator: https://qxmd.com/calculate/calculator_287/pre-test-probability-of-cad-cad-consortium

At this point we have what is known as the pre-test probability. We are going to perform a test (12 lead, or maybe a 16 lead or even a troponin) and use that information to update our best guess at the probability of this patient having ACS and arrive at a post-test probability.

Updating your views based on new information is the heart of something known as Bayes’ Rule (or Bayesian Inference, Bayes’ Theorem or Bayesian Analysis or even Bayesian Updating). There are probably subtle differences between all of the names, but I don’t really know them.

Thomas Bayes was a minister and sometime in the 1740’s he got curious about if he could predict future events based only on past events. He did some experiments with billiard balls and wrote a few pamphlets about his theories but it would not be until after his death that his ideas gained some traction. Back in those days a pamphlet was like a blog.

Bayes sort of posited (it was really refined after his death) that if you take a pre-test probability (Like a 50% guess that this is ACS) and modify it by the results of the test you’ll get the post-test probability. But things are not quite that simple because almost no test is perfect.

A tangent on testing.
It is easy to be tricked into thinking that a test for a disease can only have two results (or three if we include inconclusive results, but for simplicity we won’t)—the test can be positive or negative for disease. Positive is usually not a good thing, it means you have the disease. If we test the following 100 people we might get the following results —–

n equals 100

But again, no test is perfect. Few tests picks up every single person that actually has the disease resulting in some amount of false negatives. Surprisingly, tests often come back with a positive result even though a person does not have a disease, giving a false positive. When we look at test results we end up with the following possibilities:

two by two

When looking at a test for a disease we want to know a few things about it, we want to know how good the test is at two things (or more, but for the sake of brevity):

How good is the test at identifying people that actually have the disease?
How good is the test at identifying people that do NOT have the disease?

These are termed sensitivity and specificity. Everyone has a friend that is overly sensitive out there, and they think that everyone is talking about them. If your overly sensitive friend tells you that every person at the table of five next you is talking about them, but the truth is only one of the five really was talking about your friend (you know this because you are a curious person and you asked them later that night)  well, your friend still detected all the people that were talking about him, he just had four false positives as well. But he did pick up 100% of the people talking about him. Your paranoid friend has a sensitivity of 100%.
sensitivity

You explain that while the whole table was talking about people four out of the five people at the table were not specifically talking about him. Four people at that table were “false positives” for trash talking. While there was lots of trash talking going on at the table, 80% of it wasn’t directed at your friend. Your friend was not very good at knowing when people were not specifically talking about him. Actually, he kind of sounds like a paranoid asshole. You should get new friends.

specificity

Getting back on track here with that patient with chest pain….
How does this all help us? When we see a patient that looks like they are suffering from ACS and has a bunch of risk factors for ACS we are expecting that ECG to show something, to be positive for ACS or even a STEMI. You’ll notice I did not use STEMI here because a STEMI is a test of an ECG and based on voltage and the only way to really say if it is a true positive or not is if an expert agree with us. A STEMI is a heart attack but not all heart attacks are STEMIs – we should probably embrace the new paradigm of OMI/NOMI.

Before doing this ECG I would estimate that this patient has at least a 50% chance of suffering from ACS. When the ECG comes back absolutely normal where does that leave us? It certainly caused many people to change their minds.

sketch1570245471188
Question three asked: Now that you have seen the ECG and it was normal with no ischemia/infarct noted, what would you estimate the chances that this is ACS to be at now? This is all about the post test probability. This is very interesting – in the previous question 0% of respondents thought this was less than a 10% chance of being an infarct but a negative ECG changed the minds of 7.2% of respondents. Also before the ECG 35.2% of the respondents were thinking there was a >70% chance this was an infarct, the negative ECG brought that number down to just 8.7% of respondents!

The ECG for this patient is “negative” but you should be wondering just how good of a test is an ECG? How often does it have false negatives (which should scare us the most) and false positives? Is it a good test? Is it good enough to change transport decisions on? How much “weight” should a test like an ECG hold when we make these decisions? Is an ECG a true negative for ACS? Does it pick up everyone with ACS?

The ECG is going to be subject to some interpretation and certainly depends on who is reading it and what criteria is used to say ACS / No ACS (voltage, patterns like DeWinters, Gestalt?) and there is not tons of data out there on the sensitivity and specificity of ECGs for ACS, but what could find is this:

sens and spec ecg
If anyone has any newer data please let me know, this is from 2001.  (Ioannidis JP, Salem D, Chew PW, Lau J. Accuracy and clinical effect of out-of-hospital electrocardiography in the diagnosis of acute cardiac ischemia: a meta-analysis. Ann Emerg Med. 2001;37(5):461-70.)

Most people opted for the hospital that was further away but had a cath lab.

sketch1570247082343
72.2% took the further drive to cath lab, 27.8% went to the level IV.

So what should we do when confronted with a  patient that has an apparent negative result from an imperfect test?

Tune in to part III for likelihood ratios, fun with nomograms, thinking like a Bayesian and why no one is going to check my prostate.


Special thanks goes out to all the people I have driven crazy with this series (JJ, TC, JB, RC, AM, JK, and probably many more that I forgetting).

Also, this is not a perfect finished draft, it is sloppy,  but you know, I got a kid and it is conference season and I need to hit the publish button on this sucker.

 

 

The Confusion Matrix part 1: Infarcts (maybe?), Thomas Bayes, and we’d like to ask you a few questions.

It is a busy Friday night at the rural EMS service where you work as a paramedic. You’re dropping off a patient at the local level IV hospital but before you can even finish your hand-off report dispatch pages you again. “Please respond to a 58 year old male that is conscious and breathing, complaining of severe chest pain, no other information is available.”
Its been eight hours since you last got some food and you’d really like to hit the EMS lounge on the way out the door. You start to wonder, how likely is this to be a myocardial infarction just based on the dispatch info?
MI venn

On scene the patient is an overweight, 58-year old male sitting in a recliner. He looks like shit. He is profoundly diaphoretic, has Levine’s sign when showing you where his chest pain is, describing it as “crushing,” rating it at 8/10. When you ask if the pain goes anywhere he says it travels to both arms. He denies any trouble breathing or shortness of breath, but he does tell you that he feels more tired than usual after walking up a flight of stairs.

His chest pain started 12 hours ago when he was at work. He was hoping it was GERD but Zantac and tums did nothing for his pain and he thinks it probably isn’t GERD at this point. He tells you that he had an MI in 2014 and that this feels just like that one did. He got one stent placed in 2014 but can’t remember which artery it was in, he thinks he has some paperwork on it in a drawer on it somewhere.

You give him 162mg of ASA and obtain his vital signs. He has a heart rate of 88 beats per minute, manual blood pressures are done in both arms,  142/94 in the R arm and 140/92 in the L arm. He is breathing 18 times per minute and his room air oxygen saturation is 95% with a great “pleth” wave and he is afebrile at 98.4 degrees. Both lung sounds are clear to auscultation. When you palpate his chest wall and ask if that changes the pain he replies, “I’m not sure.”

He has a history of coronary artery disease, peripheral artery disease, hypertension, hypercholesterolemia and benign prostate hyperplasia. He takes 81mg of aspirin every day, atorvastatin, flomax, lisinopril, and some vitamins. He used to have some nitro but it expired and he never bothered with refilling it. He has no allergies. He is supposed to schedule a stress test with his doctor next month as part of a routine follow-up but hasn’t done it yet.

You establish an IV and as your partner gets the 12-lead ecg set up you begin to contemplate where to take this patient. You have two choices; there is a level IV hospital twelve minutes away and a level II hospital fifty eight minutes away. They are in opposite directions.

The level IV hospital has board certified EM physicians but there is no cath lab there, they do have TNKase available and can consult with cardiology at the level II. Due to thunderstorms in the area flights are grounded for the next few hours so they are not an option. If you bring this patient to the level IV and it turns out he is having an MI,  you will have to transfer him to the level II which is an hour and ten minutes away.

The level II hospital really likes to work with EMS and they came and did an in-service for your EMS agency last month about their cardiac alert protocol – they have started to perform urgent PCI on some NSTEMI patients in addition to the regular STEMI patients.  You can activate a “cardiac alert” there with nothing more than a gut feeling if you like. When you activate the “cardiac alert” a cardiologist or PA from cardiology meets you at the door, performs an I-stat troponin, gets a hand-off report and decides if the patient goes straight to the cath lab or they stay in the ED for more of a work-up.

The twelve lead comes back as a textbook normal sinus rhythm with no other changes noted – no subtle ST segment depression, no T wave inversion, no De Winter’s T waves, no hemi-blocks or anything else is noted, this is just a normal ecg. This is surprising as you were pretty certain you would see a STEMI on there. Just to be sure, you do a V4R and V7, V8, and V9 and still see no signs of infarct or ischemia on the ecg.

You get the patient loaded up in your ambulance and give him a quick squirt of nitro under the tongue. You set out the fentanyl because you rarely get patients down pain to a comfortable level with just nitro. 

Your partner yells from the front, “which hospital are we going to?”

This is part one of this article. Part two will be coming out next week and will look at the answers to these questions as well as look at the results from the dozens of readers of this blog.

14 in 13

A 14 gauge IV in a 13-year-old girl

Teresa Forson lost her job as a firefighter because she started a 14-gauge IV on a drunk 13-year-old and then lied about the circumstances surrounding the event.

The 13-year-old girl was alert and ambulatory with stable vital signs. Many people on social media defended the firefighter, feeling that termination was uncalled for, that it was excessive and that really, this was not that of a big deal. In one sense they are right, a 14-gauge IV insertion probably doesn’t hurt much more than a 20-gauge IV does and since no harm came to the patient from this incident, what is the big deal?

Intent is the big deal. Intent is what matters. Either these paramedics need some serious remediation on when large bore IVs are needed, or this was a punitive act. I can’t truly say what occurred in the back of that ambulance between the surly drunk teenager and the firefighter as I wasn’t there and I don’t have all the facts, but it sure sounds a lot like punitive medicine.

Practicing punitive medicine is indefensible. It points to low levels of emotional intelligence and poor impulse control. I certainly have had moments in my career where I have contemplated doing it to patients. When I first started in EMS, I believed that “drunks get 14’s,” and I was more than ready to plug some 14-gauge IVs into the next drunk patient I encountered. It would take a few years of working in EMS before I realized that there might be a better way to learn how to care for patients than teaching via war stories from people who had repeated their first-year twenty times over.

A lack of emotional intelligence training in healthcare education.

Healthcare education rarely teaches about soft skills like emotional intelligence. These skills will be used on almost every EMS call, on almost every shift and yet we don’t talk about them. Time is sent on garbage like the KED and taping people to plastic boards.

Emotional intelligence may not be real form of intelligence, there certainly appears to be a debate about that. It may be more pop-psychology than an actual science, but the skills and attributes emphasized by it are very real and can save or prolong a career.

Increasing emotional intelligence can change how you relate to the bullshit calls. If you have worked in EMS for some time you have probably encountered people that were extremely intelligent in the conventional sense yet had astoundingly low levels of emotional intelligence. These people are smart but they tend to explode over small things or end up doing some sort of punitive thing to a patient that ends their career.

“If your emotional abilities aren’t in hand, if you don’t have self-awareness, if you are not able to manage your distressing emotions, if you can’t have empathy and have effective relationships, then no matter how smart you are, you are not going to get very far.”
-Daniel Goleman

Emotional intelligence has four or five components to it depending on the source you read; self-awareness, self-regulation, motivation, empathy and social skills.  Each component is important but self-regulation  might be the most important when it comes to not getting fired and not fucking up your life in general.

Having impulses to punish a patient is not the problem; not being able to control the impulse is a problem. You can hate your patient, you can get pissed off at them, you can find them annoying, but then you move on and do your job like a professional. I have had more than one fantasy where I tell  my partner to pull over on the side of the road and kick a patient out of the ambulance in the middle of nowhere because they annoyed the shit out of me.

The obstacle is the way.

The impediment to action advances action. What stands in the way becomes the way.” -Marcus Aurelius

The Obstacle is The Way by Ryan Holiday is a short book that might change the way you look at the world. Anyone working in healthcare should read it. It transforms how you relate to all the bullshit encountered in healthcare.

In EMS there really are only two kinds of calls—bullshit calls and good calls.

The drunks, the pointless nursing home runs, the patients with back pain that should just harden up and deal with it, the rambling psych patients who went off their meds, the uninjured person that “just wants to be checked out” in the middle of the night, the repeated accidental life alert alarm activations, a pair of piss soaked pants rubbing on your pants, patients with shit packed under their fingernails that keep trying to touch you, drug-seekers, COPD patients smoking while on oxygen and complaining of shortness of breath, the 25-year-old male with chest pain at the jail, and the morbidly obese that are will blow out your back. These are the kinds of patients that suck the life out of healthcare provider. These are the kinds of patients that on bad day are easy to hate. You might even tell yourself that these patients are the obstacle to your happiness at this job—that if it weren’t for the bullshit calls you would be happy at work.

The bullshit calls are the obstacle and they are the way.

You can still be annoyed or pissed about these calls. I certainly am from time to time, but it happens less than it used to, and it has becme more of a passing thought than anything else. It is not a strong reaction. I may not like the patient, or I might be mad, but it is not a big deal. It does not linger; it does not ruin my day most of the time and it certainly doesn’t cause me to lose control. It is more along the lines of when I want Coke and must settle for the apologetic “is Pepsi okay?” Being annoyed about these calls doesn’t accomplish anything, being pissed off about these calls is a waste of time and energy.

Marcus Aurelius asks, “Does what happened keep you from acting with justice, generosity, self-control-sanity, prudence, honesty, humility, straightforwardness?”
No? Then brush it off and move on. If an asshole patient can control your actions, you probably are not really in control as much as you like to think you are.

Making it a practice.
If I get mad, they win.

When presented with an especially difficult patient I remind myself, I get mad, they win.

If a patient can provoke me to a point where I lose my composure, they win. Don’t get me wrong, I’ll escalate appropriately and professionally when needed; I’ll stab someone in the ass with 400mg of ketamine without a second thought and I’ll fight if I have no other choice. But I won’t act out of anger and I won’t give out punitive measures.

A drunk 13-year old girl certainly could be considered just another bullshit call. Or it could be an exercise in patience and self-regulation; it could be a lesson in managing your emotions.

“I don’t want to be at the mercy of my emotions. I want to use them, to enjoy them, and to dominate them.
-Oscar Wilde

 

The Agile EMS Manifesto

Presentation1

 

Individuals and interactions over processes and tools.
The people of the organization are the most important thing, everything else is secondary.

The second highest priority is the delivery of quality healthcare to the community.
Providing good healthcare over profits, expansion,  political jockeying,  career advancement, or being progressive. Taking people to the hospital and being nice to them is 90% of the job.

Guidelines over strict regulations.
Protocols must be guidelines that allow people to accomplish the goal of quality patient care. Protocols should not be rigid doctrine that must be followed even if the results are deleterious.

Welcoming changing practices based on new evidence and knowledge.
Evidence kills sacred cows and dogma – walk away from things that no longer serve a purpose.

Build projects around motivated individuals.
This means hiring the right people – ones that you are willing to invest in over the long term. Build a team, not a workforce.

The most efficient and effective method of conveying information is face-to-face conversation.
You must talk with the providers in your system face to face – there is no substitute.

A sustainable work lifestyle.
You can’t successfully provide quality healthcare to a community by forcing people to work overtime for months at a time and burning them out.

Continuous attention to technical skills.
Skills must be practiced regularly or they will atrophy. Skills should not be the hard part of the job; thinking should be the hard part.

Simplicity – the art of maximizing the amount of work not done – is essential.
Get rid of the bullshit;  in the documentation program and in anything else that prevents good patient care from happening. Simplify and streamline processes, remove things that suck the joy out of work.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Honest evaluations and feedback are needed at the individual and organizational level. When a measurement becomes a target, the value is lost.

 

Adapted and/or straight up plagiarized in parts from: https://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/


I am not one for manifestos. I don’t really like the word manifesto as it has taken on a meaning different than the actual definition, but it is what the original document was called when a small group of software developers that were fed up wrote the Agile Manifesto in 2001.

Make no mistakes here, this is idealistic, hell, maybe it is even unrealistic but it might be what is needed.

Ketamine: The Sex Panther.

Ketamine may do which of the following in a patient with shock:
A) Raise blood pressure
B) Decrease blood pressure
C) Not cause a change in blood pressure
D) All of the above

There are some misconceptions about ketamine in emergency medicine and specifically in EMS. Some EMS providers believe ketamine will ALWAYS raise blood pressure, acting like a vasopressor. Ketamine is a great drug but in some patients it can decrease perfusion.

 

sex panther
“Sex Panther by Odeon. 60% of the time, it works every time.”

Continue reading “Ketamine: The Sex Panther.”

Just culture is dead.

Just culture is dead.

It began as a beautiful idea but it is almost unrecognizable now. It has become something dirty and impure, a tool for power hungry people to label others and think they are doing something productive.

Just culture has become another bureaucratic policy, another mandatory training that people have to sit through while staring at bad PowerPoints and watching the clock.

If your organization thinks that embracing just culture is using an algorithm to decide if someone can be blamed for something or not, then it has already failed. Continue reading “Just culture is dead.”

A field guide to EMS social media commentary: the seven levels of reflective judgment, plus a story about my roof.

“Think about it. 7-Elevens. 7 dwarves. 7, man, that’s the number. 7 chipmunks twirlin’ on a branch, eatin’ lots of sunflowers on my uncle’s ranch. You know that old children’s tale from the sea. It’s like you’re dreamin’ about Gorgonzola cheese when it’s clearly Brie time, baby.”

I spend way too much time on EMS social media. I am fascinated by some of the comments that are posted – the dismissal of science and rational thought, the flawed logic, and the ignorant certainty that abound in the comments section provides a window into the flawed inner workings of the human brain.

 I recently stumbled onto the reflective judgment model by King and Kitchener. It seems to be a decent tool for exploring and identifying the behaviors in EMS social media commentary and EMS in general. Reflective judgment is the process of thinking about how you know what you know and how true those facts are. There are seven levels of reflective judgment proposed by King and Kitchener in their 1994 work, Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults.

Level 1: “I’ve seen it work.”
This is the land where anecdote is king and correlation is causation. Continue reading “A field guide to EMS social media commentary: the seven levels of reflective judgment, plus a story about my roof.”

Something About Nothing: ROSC is a Meaningless Outcome (and The 2018 ACLS Update).

ROSC is not an outcome that matters.

“You have to get ROSC before you get anything else.”

“If patients don’t get ROSC they can’t live, so anything that increases ROSC is giving that patient a chance.”

It makes sense to think that ROSC is an important outcome, at least on a superficial level. Patients need to get ROSC at some point if they are going to have a  good neurologic outcome. It is true, but it is a half-truth and unless it is examined health care providers will continue to administer ineffective and perhaps harmful treatments.

Resuscitation is more than the sum of its parts. Continue reading “Something About Nothing: ROSC is a Meaningless Outcome (and The 2018 ACLS Update).”

Why you need a stroke ambulance by Chet J. Reilly (guest post)

Questioning the benefits of stroke ambulances seems to be in vogue lately in certain circles. In the interest of self-promotion these individuals love to point out the perceived shortcomings of others. As an EMS leader and agent of change, I feel I need to bring some clarity to this issue. I will outline the many benefits of having a stroke ambulance in your community below. Continue reading “Why you need a stroke ambulance by Chet J. Reilly (guest post)”

The Empty Suits of EMS.

With absolute certainty the EMS empty suit proclaims that a college degree will fix all the issues EMS faces both today and in the future. Strangely, many of the proponents of a degree requirement do not even have a college degree themselves. They honestly believe it is a simple equation of supply versus demand. Degrees will cause there to be fewer paramedics and therefore increased demand for paramedics, which will fix everything. Complex problems rarely have simple answers.

Raising the barriers to entry and decreasing the number of paramedics won’t automatically equal higher pay for paramedics, it might just create more bastardized dumbed-down version of paramedic to skirt the regulations- think EMT-Intermediate,* Advanced EMT–Critical Care, Cardiac-EMT, Enhanced EMT, IEMT and all the other alphabet-soup flavors of ALS-lite that exist  for communities that can’t or won’t pay for paramedics to be on staff or volunteers that don’t want undertake that much education to help out their community. [1]

Continue reading “The Empty Suits of EMS.”