emsqaqi

The scenario

Your EMS agency has decided to test the staff for COVID-19 antibodies. The results of each person will be kept private — you will only find out your results, not those of others.

Your county has 20,000 people in it. The public health department performed 10,000 tests for COVID-19 infection with a viral PCR tests last week. There have been no new cases in the county since then. Out of the 10,000 tests done, 500 people were positive. Assume that this test is 99.999% valid and were not concerned about false positives or false negatives with this test. You just need to take this number at face value for now. 500 out of 10,000 people had COVID in your county.

You remember feeling sick in February, but because testing was not available then, you were never tested. You don’t know if you actually had it or not this past winter. A few of your workers think they might have been sick then, but most were not sick last winter. You did not have a true exposure that you know of at work. You were wearing the proper PPE every time you treated a patient with COVID-19 this spring at work and you are a diligent hand-washer.

Two days later your COVID 19 Antibody test comes back positive.

But you are a curious person. Your mind creates countless what-if scenarios. You decide to get the antibody test done—data is showing that perhaps up to 40% of those with COVID-19 never have symptoms or are paucisymptomatic.

The package insert for the antibody tests your agency uses states it is 95% sensitive and 95% specific. That seems pretty reliable, doesn’t it?

What are the chances that you really did have COVID-19 based on the antibody test results?
A) 99% chance you can trust the results
B) 95% chance you can trust the results
C) 63% chance you can trust the results
D) 50% chance you can trust the results
E) 5% chance you can trust the results

What would you say the chances are that you actually had COVID 19? In order to really understand what these results mean we need to discuss a few terms first.

Testing:
Tests should accurately discriminate between those who have a disease and those who do not have a disease. It’s tempting to think of things in binary terms here; either they have the disease and test positive or they don’t have the disease and test negative. But that is not how things work. Complex problems rarely have simple answers.

Test results:
When testing for a disease or condition there are four possible results, aside from things like inconclusive results, which for the sake of clarity we are not going to discuss.

True positive: persons with the disease that test positive for the disease. 
True negative: persons without the disease that test negative for the disease.
False positive: persons without the disease but they tested positive.
False negative: persons with disease but the test failed to detect it.

We can take the four results and put them in a table called a 2 x 2 contingency table, also know as a confusion matrix.

Prevalence: What percentage of the population has this condition? Take 1000 people and see how many of them have the disease you’re testing for and that is a good estimate of prevalence. Of course, you need to make sure the sample is representative of the population you are testing.

Specificity: How well a test can identify those who do not have the disease. A false negative is probably the scariest result in healthcare. That nagging cough you’ve had all week—is it just allergies or is the start of COVID? You are supposed to work in a few days and don’t want to give your coworkers the ‘Rona.

Specificity is important because it tells us how much faith we can put in the negative result from a test -how well does the test identify those who DON’T have the disease.

To calculate this value take the number of true negatives and divide it by the number of true negatives and false positives. Put another way the number of negatives divided by the number of all the people who really don’t have the disease.

Sensitivity: How well a test can identify those with the disease. To calculate this value, divide the number of true positives by the number of true positives and false negatives (people the test missed to detect) combined. You could design a test that is 100% sensitive, but the rates of false positives could be staggering as well. 100% sensitivity is best explained by thinking of your paranoid friend who thinks every thing everyone says is about them. They are 100% sensitive, but frequently misidentify things as being about them when they aren’t.

Half of the results will be false positives.
Testing in this scenario is the same as tossing a coin. Does this make the results meaningless. Maybe? It depends on a few other things.

If the test becomes better, if the sensitivity or specificity is increased, it changes how much we can trust the results. There are several examples that follow. A word of caution should be said; many manufacturers put sensitivity and specificity numbers in their information that make the tests look near perfect – real life often shows that these numbers are much lower than initially claimed.

A great test for a disease with a prevalence 5% is pretty reliable. But what happens if the disease is not quite as commonplace as we thought? Even if we have the new, improved test, dropping the prevalence from 5% to 1% we end up with results that are essentially meaningless again, at 50/50.

Putting it all together: Bayes Theorem.
We want to know how likely is it that someone has a disease after a test, in this case we want to know how likely is it that someone really has COVID-19 antibodies and not a false positive.

With two pieces of information we can compute the chances that someone has a disease based on a test.

Remembering the contingency table from earlier, tests can produce both an actual positive and a false positive. How do we figure out if someone is a false positive or if they are a true positive?

We need to know two pieces of information before we can do this.

Pre-test probability:
What are the chances a person has the disease before any testing is done? The answer could also be a damn good guess that the febrile patient with a pericardial friction rub heard on auscultation is going to end up having pericarditis.

When there is no other information available prevalence can be used to establish this. If there is more information available this can be a well reasoned or evidence based number, but it might just be a well educated guessing.

If all things are equal, and there is no more information to add to the equation, then prevalence in a population will be the pre-test probability.

If a random person was snatched off the street and you knew nothing about them, you would say their pre-test is the same as prevalence, roughly 3-5% for having had COVID. If the same person was in the ICU on oxygen, proned, and had ARDS following a flu-like illness last week their chances are much higher than 5%, it seems that something like a 50% pre-test probability would be closer to the truth.

Taking a healthcare worker who had a flu like illness this winter but never got tested for COVID-19 the pretest probability is certainly higher than the prevalence for all people. The more information we can add to this scenario, the more accurately we can determine a pre-test probability.

Likelihood ratio:
How certain can we be that a positive result is an actual positive and not a false positive?

Likelihood ratios are broken down into a positive likelihood ratio and a negative likelihood ratios. How much should test results change our beliefs about reality? How much does a positive or negative result change how we view our initial diagnosis?

If we are fairly certain someone is having an AMI but their ECG is normal, how much do we adjust our beliefs? Not that much. The reason being that a negative ECG is not that powerful for ruling out ischemia.

To calculate the positive likelihood ration you need to know how often the test produces false positives. Taking the true positive rate and dividing it by the false positive rate tells you how much trust should be placed in a positive result. The negative likelihood ratio is the opposite – what do we do with a negative result?

To calculate the negative likelihood ratio you need to know how good the test is at detecting those who do not have the disease (specificity). Divide the false negative rate by the true negative rate to find out how much you should trust a negative result.

The Fagan Nomogram:
Nomograms are three column charts. Drawing a line between two columns of numbers leads to a third column of numbers.

You want to know what your results of the COVID 19 antibody test mean rather than some random person’s results. All things being equal you start out with a 3-5% chance of having this disease, based on prevalence alone- but are all things equal? Probably not. If you can add new information to the question you answer changes. Updating beliefs as new information is presented is the heart of Bayes Theorem.

If you live in the middle of nowhere, off the grid and haven’t seen another person in six months your pre-test probability is essentially zero, which if you multiply anything by zero, the answer is zero. If you a healthcare provider and have worked with numerous COVID-19 positive patients in the past months you are probably more likely to have COVID-19 than the average person.

The best guesses for the sensitivity and specificity COVID-19 antibody tests range from 95-99% sensitivity and 95-99% specificity. The test manufacturers data should be taken with large grains of salt; many of them have not performed as advertised. When the likelihood ratios are calculated, using 95% we end up at a positive likelihood ratio of 19 and a negative likelihood ratio of 0.05.

To be honest, I’m not even sure how to multiply a percent by ratio. The good news is you do not have to be good at math to understand concepts. Let the Fagan Nomogram do the work for you.

The answers to the question about YOUR chances:
What follows is three different scenarios. The first one uses 5% prevalence as the pre-test probability, then moving on to 20% and 30% pre-test probability. The first nomogram is the answer to our initial scenario.

Drawing a line from 5% pre-test probability across the likelihood ratio of 19 leads us to 50% post-test probability. This is the same number we arrived at with the frequency tree earlier in the article with 47 positives and 47 false positives. What if we increase the pre-test probability though?

Increasing the pre-test probability a bit more makes the positive results much more trustworthy.

On the other hand, it seems things have become less certain the more we know. COVID-19 presents and ever moving target. Any time we begin to feel like we are getting our footing on solid ground the carpet is pulled out from under our feet. A recent study demonstrated that 40% of those confirmed to have had a previous COVID-19 infection did not have detectable levels of antibodies in their blood several months post infection.

Switching from binary thinking, from thinking either someone has a disease or they do not, to a probabilistic approach is hard. The world becomes less black and white, answers are harder to find, and at times it can be exhausting. But as clinicians we should try to align our beliefs with reality as much as possible. Getting closer to the truth often involves embracing uncertainty.

Thanks to JK and RC for holding my hand with some of the math in this post.

Fear is the mind-killer. Fear is the little-death that brings total obliteration.

Moments of terror alternating with calm acceptance. What can be done? Nothing? Do the best you can, the rest is out of your hands.

“It’s gon’ rain down like black hell.”

How much VUCA can you endure and still be calm? Remember – what would Marcus do?

Don’t believe yourself, there is always time to slow down and think.

Fear leads to system 1. System 1 leads to disaster. Check and re-check.  Trust no one, especially yourself.

Do you really know that or do you just think you know that? Buggy knowledge will take you all the way to the scene of the crash.

Now is not the time to “think so.” You must know it in and out. Chauffer knowledge won’t keep you safe anymore.

This is the Red Queen’s race.

Those who can adapt will persevere.

Most of your fear is about a future that is yet to happen. The present is actually not too bad right now.

Check another day off towards that 14-days to freedom (no, not that kind) and sign off for the night. Halfway there.

[Brain Dump Complete – thanks to Glen Danzig and one from Buffy for the accompaniment tonight. Will this continue? Tune-in. “]

God Damn: https://www.youtube.com/watch?v=r_9gZEMQ2No&list=PLrCDgKRSK58vTo5b2dZLaNVfVFLcPFD-R&index=57

This is part two in a three part series.

Part one of this three part series asked a few questions about what the probabilities are that a patient with chest pain is having a myocardial infarction and the answers are below in this blog.

Lots of people seek emergent medical treatment for chest pain. Most chest pain ends up being something other than ACS; things like pneumonia, anxiety, costochondritis, referred pain from an organ, a pulmonary embolism – the differential is lengthy and everyone knows it.

The first question in the previous post asks about how likely it is that the patient is having ACS based only on the dispatch information.

This first question is all about prevalence – how many people in a given population have this condition. The odds that this is ACS are fairly low just based on the dispatch information. The best data on this comes from patients with chest pain seeking medical treatment at an ED that are eventually diagnosed with ACS. Between 13% and 14% of patients seeking emergent treatment for chest pain are eventually diagnosed with ACS.

The way to read this chart might be a bit confusing at first. If you look at the first column, 21.8% of the people who answered this question felt there was less than a 10% chance this was an infarct based just on the dispatch info. The blue column shows that 28.2% of people who answered the question thought that the chances were between 11-30% that this was ACS based just on dispatch info.

After getting on scene in the scenario and seeing the patient a lot people changed their answers to a higher probability that this is ACS.

In light of new evidence (typical chest pain, arm pain, gestalt, etc.) we should update our views of the probability. and this goes both ways—if the patient was busy playing angry birds and looked up at you and smiled while they said their pain was a 10 out of 10 you might update your estimates of probabilities in the opposite direction.

In a few cases this can be accomplished using tools or this can be based on an educated guess if there aren’t validated metrics out there. I fathomed a guess of around a 50% probability and I think it was pretty close based on this tool. Pre-test probability calculator: https://qxmd.com/calculate/calculator_287/pre-test-probability-of-cad-cad-consortium

At this point we have what is known as the pre-test probability. We are going to perform a test (12 lead, or maybe a 16 lead or even a troponin) and use that information to update our best guess at the probability of this patient having ACS and arrive at a post-test probability.

Updating your views based on new information is the heart of something known as Bayes’ Rule (or Bayesian Inference, Bayes’ Theorem or Bayesian Analysis or even Bayesian Updating). There are probably subtle differences between all of the names, but I don’t really know them.

Thomas Bayes was a minister and sometime in the 1740’s he got curious about if he could predict future events based only on past events. He did some experiments with billiard balls and wrote a few pamphlets about his theories but it would not be until after his death that his ideas gained some traction. Back in those days a pamphlet was like a blog.

Bayes sort of posited (it was really refined after his death) that if you take a pre-test probability (Like a 50% guess that this is ACS) and modify it by the results of the test you’ll get the post-test probability. But things are not quite that simple because almost no test is perfect.

A tangent on testing.
It is easy to be tricked into thinking that a test for a disease can only have two results (or three if we include inconclusive results, but for simplicity we won’t)—the test can be positive or negative for disease. Positive is usually not a good thing, it means you have the disease. If we test the following 100 people we might get the following results —–

But again, no test is perfect. Few tests picks up every single person that actually has the disease resulting in some amount of false negatives. Surprisingly, tests often come back with a positive result even though a person does not have a disease, giving a false positive. When we look at test results we end up with the following possibilities:

When looking at a test for a disease we want to know a few things about it, we want to know how good the test is at two things (or more, but for the sake of brevity):

How good is the test at identifying people that actually have the disease?
How good is the test at identifying people that do NOT have the disease?

These are termed sensitivity and specificity. Everyone has a friend that is overly sensitive out there, and they think that everyone is talking about them. If your overly sensitive friend tells you that every person at the table of five next you is talking about them, but the truth is only one of the five really was talking about your friend (you know this because you are a curious person and you asked them later that night)  well, your friend still detected all the people that were talking about him, he just had four false positives as well. But he did pick up 100% of the people talking about him. Your paranoid friend has a sensitivity of 100%.

You explain that while the whole table was talking about people four out of the five people at the table were not specifically talking about him. Four people at that table were “false positives” for trash talking. While there was lots of trash talking going on at the table, 80% of it wasn’t directed at your friend. Your friend was not very good at knowing when people were not specifically talking about him. Actually, he kind of sounds like a paranoid asshole. You should get new friends.

Getting back on track here with that patient with chest pain….
How does this all help us? When we see a patient that looks like they are suffering from ACS and has a bunch of risk factors for ACS we are expecting that ECG to show something, to be positive for ACS or even a STEMI. You’ll notice I did not use STEMI here because a STEMI is a test of an ECG and based on voltage and the only way to really say if it is a true positive or not is if an expert agree with us. A STEMI is a heart attack but not all heart attacks are STEMIs – we should probably embrace the new paradigm of OMI/NOMI.

Before doing this ECG I would estimate that this patient has at least a 50% chance of suffering from ACS. When the ECG comes back absolutely normal where does that leave us? It certainly caused many people to change their minds.

The ECG for this patient is “negative” but you should be wondering just how good of a test is an ECG? How often does it have false negatives (which should scare us the most) and false positives? Is it a good test? Is it good enough to change transport decisions on? How much “weight” should a test like an ECG hold when we make these decisions? Is an ECG a true negative for ACS? Does it pick up everyone with ACS?

The ECG is going to be subject to some interpretation and certainly depends on who is reading it and what criteria is used to say ACS / No ACS (voltage, patterns like DeWinters, Gestalt?) and there is not tons of data out there on the sensitivity and specificity of ECGs for ACS, but what could find is this:

Most people opted for the hospital that was further away but had a cath lab.

So what should we do when confronted with a  patient that has an apparent negative result from an imperfect test?

Tune in to part III for likelihood ratios, fun with nomograms, thinking like a Bayesian and why no one is going to check my prostate.

---

Special thanks goes out to all the people I have driven crazy with this series (JJ, TC, JB, RC, AM, JK, and probably many more that I forgetting).

Also, this is not a perfect finished draft, it is sloppy,  but you know, I got a kid and it is conference season and I need to hit the publish button on this sucker.

 

 

A 14 gauge IV in a 13-year-old girl

Teresa Forson lost her job as a firefighter because she started a 14-gauge IV on a drunk 13-year-old and then lied about the circumstances surrounding the event.

The 13-year-old girl was alert and ambulatory with stable vital signs. Many people on social media defended the firefighter, feeling that termination was uncalled for, that it was excessive and that really, this was not that of a big deal. In one sense they are right, a 14-gauge IV insertion probably doesn’t hurt much more than a 20-gauge IV does and since no harm came to the patient from this incident, what is the big deal?

Intent is the big deal. Intent is what matters. Either these paramedics need some serious remediation on when large bore IVs are needed, or this was a punitive act. I can’t truly say what occurred in the back of that ambulance between the surly drunk teenager and the firefighter as I wasn’t there and I don’t have all the facts, but it sure sounds a lot like punitive medicine.

Practicing punitive medicine is indefensible. It points to low levels of emotional intelligence and poor impulse control. I certainly have had moments in my career where I have contemplated doing it to patients. When I first started in EMS, I believed that “drunks get 14’s,” and I was more than ready to plug some 14-gauge IVs into the next drunk patient I encountered. It would take a few years of working in EMS before I realized that there might be a better way to learn how to care for patients than teaching via war stories from people who had repeated their first-year twenty times over.

A lack of emotional intelligence training in healthcare education.

Healthcare education rarely teaches about soft skills like emotional intelligence. These skills will be used on almost every EMS call, on almost every shift and yet we don’t talk about them. Time is sent on garbage like the KED and taping people to plastic boards.

Emotional intelligence may not be real form of intelligence, there certainly appears to be a debate about that. It may be more pop-psychology than an actual science, but the skills and attributes emphasized by it are very real and can save or prolong a career.

Increasing emotional intelligence can change how you relate to the bullshit calls. If you have worked in EMS for some time you have probably encountered people that were extremely intelligent in the conventional sense yet had astoundingly low levels of emotional intelligence. These people are smart but they tend to explode over small things or end up doing some sort of punitive thing to a patient that ends their career.

“If your emotional abilities aren’t in hand, if you don’t have self-awareness, if you are not able to manage your distressing emotions, if you can’t have empathy and have effective relationships, then no matter how smart you are, you are not going to get very far.”
-Daniel Goleman

Emotional intelligence has four or five components to it depending on the source you read; self-awareness, self-regulation, motivation, empathy and social skills.  Each component is important but self-regulation  might be the most important when it comes to not getting fired and not fucking up your life in general.

Having impulses to punish a patient is not the problem; not being able to control the impulse is a problem. You can hate your patient, you can get pissed off at them, you can find them annoying, but then you move on and do your job like a professional. I have had more than one fantasy where I tell  my partner to pull over on the side of the road and kick a patient out of the ambulance in the middle of nowhere because they annoyed the shit out of me.

The obstacle is the way.

The impediment to action advances action. What stands in the way becomes the way.” -Marcus Aurelius

The Obstacle is The Way by Ryan Holiday is a short book that might change the way you look at the world. Anyone working in healthcare should read it. It transforms how you relate to all the bullshit encountered in healthcare.

In EMS there really are only two kinds of calls—bullshit calls and good calls.

The drunks, the pointless nursing home runs, the patients with back pain that should just harden up and deal with it, the rambling psych patients who went off their meds, the uninjured person that “just wants to be checked out” in the middle of the night, the repeated accidental life alert alarm activations, a pair of piss soaked pants rubbing on your pants, patients with shit packed under their fingernails that keep trying to touch you, drug-seekers, COPD patients smoking while on oxygen and complaining of shortness of breath, the 25-year-old male with chest pain at the jail, and the morbidly obese that are will blow out your back. These are the kinds of patients that suck the life out of healthcare provider. These are the kinds of patients that on bad day are easy to hate. You might even tell yourself that these patients are the obstacle to your happiness at this job—that if it weren’t for the bullshit calls you would be happy at work.

The bullshit calls are the obstacle and they are the way.

You can still be annoyed or pissed about these calls. I certainly am from time to time, but it happens less than it used to, and it has becme more of a passing thought than anything else. It is not a strong reaction. I may not like the patient, or I might be mad, but it is not a big deal. It does not linger; it does not ruin my day most of the time and it certainly doesn’t cause me to lose control. It is more along the lines of when I want Coke and must settle for the apologetic “is Pepsi okay?” Being annoyed about these calls doesn’t accomplish anything, being pissed off about these calls is a waste of time and energy.

Marcus Aurelius asks, “Does what happened keep you from acting with justice, generosity, self-control-sanity, prudence, honesty, humility, straightforwardness?”
No? Then brush it off and move on. If an asshole patient can control your actions, you probably are not really in control as much as you like to think you are.

Making it a practice.
If I get mad, they win.

When presented with an especially difficult patient I remind myself, I get mad, they win.

If a patient can provoke me to a point where I lose my composure, they win. Don’t get me wrong, I’ll escalate appropriately and professionally when needed; I’ll stab someone in the ass with 400mg of ketamine without a second thought and I’ll fight if I have no other choice. But I won’t act out of anger and I won’t give out punitive measures.

A drunk 13-year old girl certainly could be considered just another bullshit call. Or it could be an exercise in patience and self-regulation; it could be a lesson in managing your emotions.

“I don’t want to be at the mercy of my emotions. I want to use them, to enjoy them, and to dominate them.
-Oscar Wilde

 

 

Individuals and interactions over processes and tools.
The people of the organization are the most important thing, everything else is secondary.

The second highest priority is the delivery of quality healthcare to the community.
Providing good healthcare over profits, expansion,  political jockeying,  career advancement, or being progressive. Taking people to the hospital and being nice to them is 90% of the job.

Guidelines over strict regulations.
Protocols must be guidelines that allow people to accomplish the goal of quality patient care. Protocols should not be rigid doctrine that must be followed even if the results are deleterious.

Welcoming changing practices based on new evidence and knowledge.
Evidence kills sacred cows and dogma – walk away from things that no longer serve a purpose.

Build projects around motivated individuals.
This means hiring the right people – ones that you are willing to invest in over the long term. Build a team, not a workforce.

The most efficient and effective method of conveying information is face-to-face conversation.
You must talk with the providers in your system face to face – there is no substitute.

A sustainable work lifestyle.
You can’t successfully provide quality healthcare to a community by forcing people to work overtime for months at a time and burning them out.

Continuous attention to technical skills.
Skills must be practiced regularly or they will atrophy. Skills should not be the hard part of the job; thinking should be the hard part.

Simplicity – the art of maximizing the amount of work not done – is essential.
Get rid of the bullshit;  in the documentation program and in anything else that prevents good patient care from happening. Simplify and streamline processes, remove things that suck the joy out of work.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Honest evaluations and feedback are needed at the individual and organizational level. When a measurement becomes a target, the value is lost.

 

Adapted and/or straight up plagiarized in parts from: https://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/

---

I am not one for manifestos. I don’t really like the word manifesto as it has taken on a meaning different than the actual definition, but it is what the original document was called when a small group of software developers that were fed up wrote the Agile Manifesto in 2001.

Make no mistakes here, this is idealistic, hell, maybe it is even unrealistic but it might be what is needed.

Ketamine may do which of the following in a patient with shock:
A) Raise blood pressure
B) Decrease blood pressure
C) Not cause a change in blood pressure
D) All of the above

There are some misconceptions about ketamine in emergency medicine and specifically in EMS. Some EMS providers believe ketamine will ALWAYS raise blood pressure, acting like a vasopressor. Ketamine is a great drug but in some patients it can decrease perfusion.

 

Continue reading “Ketamine: The Sex Panther.” →