The confusion matrix part II: the probability of the posterior depends on the sensitivity, your crazy friends that ruins everything, and would you like a pamphlet?

This is part two in a three part series.

Part one of this three part series asked a few questions about what the probabilities are that a patient with chest pain is having a myocardial infarction and the answers are below in this blog.

Lots of people seek emergent medical treatment for chest pain. Most chest pain ends up being something other than ACS; things like pneumonia, anxiety, costochondritis, referred pain from an organ, a pulmonary embolism – the differential is lengthy and everyone knows it.

The first question in the previous post asks about how likely it is that the patient is having ACS based only on the dispatch information.

This first question is all about prevalence – how many people in a given population have this condition. The odds that this is ACS are fairly low just based on the dispatch information. The best data on this comes from patients with chest pain seeking medical treatment at an ED that are eventually diagnosed with ACS. Between 13% and 14% of patients seeking emergent treatment for chest pain are eventually diagnosed with ACS.

question 1 draw
Question 1 was: With only the dispatch information provided, what do you think the chances are that this patient is having a myocardial infarction? 

The way to read this chart might be a bit confusing at first. If you look at the first column, 21.8% of the people who answered this question felt there was less than a 10% chance this was an infarct based just on the dispatch info. The blue column shows that 28.2% of people who answered the question thought that the chances were between 11-30% that this was ACS based just on dispatch info.

After getting on scene in the scenario and seeing the patient a lot people changed their answers to a higher probability that this is ACS.

sketch1570244644166
Question 2: With only the dispatch information provided, what do you think the chances are that this patient is having a myocardial infarction? This is the pre-test probability. At this point no one thought the chances were less than 10% that this was an infarct.

In light of new evidence (typical chest pain, arm pain, gestalt, etc.) we should update our views of the probability. and this goes both ways—if the patient was busy playing angry birds and looked up at you and smiled while they said their pain was a 10 out of 10 you might update your estimates of probabilities in the opposite direction.

In a few cases this can be accomplished using tools or this can be based on an educated guess if there aren’t validated metrics out there. I fathomed a guess of around a 50% probability and I think it was pretty close based on this tool. Pre-test probability calculator: https://qxmd.com/calculate/calculator_287/pre-test-probability-of-cad-cad-consortium

At this point we have what is known as the pre-test probability. We are going to perform a test (12 lead, or maybe a 16 lead or even a troponin) and use that information to update our best guess at the probability of this patient having ACS and arrive at a post-test probability.

Updating your views based on new information is the heart of something known as Bayes’ Rule (or Bayesian Inference, Bayes’ Theorem or Bayesian Analysis or even Bayesian Updating). There are probably subtle differences between all of the names, but I don’t really know them.

Thomas Bayes was a minister and sometime in the 1740’s he got curious about if he could predict future events based only on past events. He did some experiments with billiard balls and wrote a few pamphlets about his theories but it would not be until after his death that his ideas gained some traction. Back in those days a pamphlet was like a blog.

Bayes sort of posited (it was really refined after his death) that if you take a pre-test probability (Like a 50% guess that this is ACS) and modify it by the results of the test you’ll get the post-test probability. But things are not quite that simple because almost no test is perfect.

A tangent on testing.
It is easy to be tricked into thinking that a test for a disease can only have two results (or three if we include inconclusive results, but for simplicity we won’t)—the test can be positive or negative for disease. Positive is usually not a good thing, it means you have the disease. If we test the following 100 people we might get the following results —–

n equals 100

But again, no test is perfect. Few tests picks up every single person that actually has the disease resulting in some amount of false negatives. Surprisingly, tests often come back with a positive result even though a person does not have a disease, giving a false positive. When we look at test results we end up with the following possibilities:

two by two

When looking at a test for a disease we want to know a few things about it, we want to know how good the test is at two things (or more, but for the sake of brevity):

How good is the test at identifying people that actually have the disease?
How good is the test at identifying people that do NOT have the disease?

These are termed sensitivity and specificity. Everyone has a friend that is overly sensitive out there, and they think that everyone is talking about them. If your overly sensitive friend tells you that every person at the table of five next you is talking about them, but the truth is only one of the five really was talking about your friend (you know this because you are a curious person and you asked them later that night)  well, your friend still detected all the people that were talking about him, he just had four false positives as well. But he did pick up 100% of the people talking about him. Your paranoid friend has a sensitivity of 100%.
sensitivity

You explain that while the whole table was talking about people four out of the five people at the table were not specifically talking about him. Four people at that table were “false positives” for trash talking. While there was lots of trash talking going on at the table, 80% of it wasn’t directed at your friend. Your friend was not very good at knowing when people were not specifically talking about him. Actually, he kind of sounds like a paranoid asshole. You should get new friends.

specificity

Getting back on track here with that patient with chest pain….
How does this all help us? When we see a patient that looks like they are suffering from ACS and has a bunch of risk factors for ACS we are expecting that ECG to show something, to be positive for ACS or even a STEMI. You’ll notice I did not use STEMI here because a STEMI is a test of an ECG and based on voltage and the only way to really say if it is a true positive or not is if an expert agree with us. A STEMI is a heart attack but not all heart attacks are STEMIs – we should probably embrace the new paradigm of OMI/NOMI.

Before doing this ECG I would estimate that this patient has at least a 50% chance of suffering from ACS. When the ECG comes back absolutely normal where does that leave us? It certainly caused many people to change their minds.

sketch1570245471188
Question three asked: Now that you have seen the ECG and it was normal with no ischemia/infarct noted, what would you estimate the chances that this is ACS to be at now? This is all about the post test probability. This is very interesting – in the previous question 0% of respondents thought this was less than a 10% chance of being an infarct but a negative ECG changed the minds of 7.2% of respondents. Also before the ECG 35.2% of the respondents were thinking there was a >70% chance this was an infarct, the negative ECG brought that number down to just 8.7% of respondents!

The ECG for this patient is “negative” but you should be wondering just how good of a test is an ECG? How often does it have false negatives (which should scare us the most) and false positives? Is it a good test? Is it good enough to change transport decisions on? How much “weight” should a test like an ECG hold when we make these decisions? Is an ECG a true negative for ACS? Does it pick up everyone with ACS?

The ECG is going to be subject to some interpretation and certainly depends on who is reading it and what criteria is used to say ACS / No ACS (voltage, patterns like DeWinters, Gestalt?) and there is not tons of data out there on the sensitivity and specificity of ECGs for ACS, but what could find is this:

sens and spec ecg
If anyone has any newer data please let me know, this is from 2001.  (Ioannidis JP, Salem D, Chew PW, Lau J. Accuracy and clinical effect of out-of-hospital electrocardiography in the diagnosis of acute cardiac ischemia: a meta-analysis. Ann Emerg Med. 2001;37(5):461-70.)

Most people opted for the hospital that was further away but had a cath lab.

sketch1570247082343
72.2% took the further drive to cath lab, 27.8% went to the level IV.

So what should we do when confronted with a  patient that has an apparent negative result from an imperfect test?

Tune in to part III for likelihood ratios, fun with nomograms, thinking like a Bayesian and why no one is going to check my prostate.


Special thanks goes out to all the people I have driven crazy with this series (JJ, TC, JB, RC, AM, JK, and probably many more that I forgetting).

Also, this is not a perfect finished draft, it is sloppy,  but you know, I got a kid and it is conference season and I need to hit the publish button on this sucker.

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s