How did the Machines figure that out and what difference will it make?
With physician burnout rates over 60% these days and physician turnover everywhere, is this ability to flag you as a Flight Risk useful at all?
In this post, let's explore the following questions
- What is Machine Learning and why is it so accurate?
- What will your leaders do with this red flag? Will it do anything to help you or them take any new actions?
- How many ways can surveillance like this be misused?
- And I will take a first pass at all the questions this new development raises.
~~~~~~~~~~~~
This blog post is based on this article
which reports on this study
~~~~~~~~~~~~
What is Machine Learning?
What a great question! At this point in human/computer evolution it is vital that everyone understand this term. So let me give you my understanding of the term.
The official Microsoft definition of Machine Learning says:
"Machine learning is an application of AI. It’s the process of using mathematical models of data to help a computer learn without direct instruction. This enables a computer system to continue learning and improving on its own, based on experience."
The biggest challenge is we don't know how Machine Learning works. What the computer is doing and how it is learning is actually unknowable by a human. The machine (a computer) is doing the learning on its own. It can't explain it to you and if it tried, humans would most likely be incapable of understanding.
============
Learn More from Our 40,000 Doctor Experience:
Burnout's highest and best use
Live Burnout Prevention Training
Meet our team of certified physician coaches
The fast track to a much better job
============
Even though you can't know how it works, Machine Learning is shockingly accurate at predicting all sorts of funky things.
Machine learning is often used to predict an outcome that looks unpredictable on the surface - to us humans.
This study found the machines able to predict which doctors would quit their job with a 97% accuracy, mostly by looking at raw data they entered into their EMR program at work.
How do the machines make these predictions?
It is actually quite simple.
They are trained to do so.
A QUICK MACHINE LEARNING TUTORIAL:
How they train the machines.
The work flow of a Machine Learning experiment begins when you pick something you want to be able to predict.
QUERY = "Which of our employee physicians will quit in the next six months?"
That sounds like something a physician leader would want to know. (We will get to the questions on what they would do with this information, below this tutorial)
Next:
Grab a bunch of data that the subjects of the query throw off in the natural course of their existence. In this case the query was about physicians, so they tossed in all the EHR input data, including EHR use time and clinical productivity measures, physician demographics including age and length of employment.
NEXT:
Identify a bunch of subjects with the outcome you are looking to predict. "Doctors who quit their job in the last 10 years." Gather a the same data on a much larger sample of physicians who did not quit in that time frame as control subjects - "stayers". For the purposes of this tutorial, please imagine the group of "quitters" has an N = 50.
NEXT:
Divide the list of "quitters" and the control group in half and let the training begin.
Even though Machine Learning is powerful and mysterious and unknowable, HOW they do it is simple to wrap your (human) brain around.
You use the first half of the list to train the machines to find the second half of the list on their own.
There are only two steps. Here's how it works.
1) TRAINING:
Select the data from half of the "quitters" (N=25) and half of the controls and dump it in the hopper for round one. Make sure you keep the remaining 25 quitters' data and half of the controls out for this training round. We will use them to test the algorithm in the next step.Tell the computers, "These 25 doctors (the "quitters") have something in common. Please identify the pattern that they share and how they are different from the rest."
That is all the instruction the machines need. Then the magic begins. Give the machines some time, space and AC current to crunch the bits and bytes.
2) TESTING:
Then you dump the remaining 25 "quitter" data and the rest of the control data into the machines and say, "Which of these doctors share the pattern you identified in the TRAINING step."
- The machines tell you which doctors have the pattern they identified in round one.
- You check to see if those names are on the list of doctors who actually quit in real life.
- Calculate the accuracy of the prediction.
You have to admit ...
97% is a shockingly high accuracy rate compared to your boss's estimate of whether or not you will leave in the next six months!
And I know, we both want to understand how this can be done. I am sorry, because it is Machine Learning, you will need to let that urge to understand go. No human can know, and no machine can explain it to you.
It is important to observe, this whole topic of Machine Learning is a classic example "association and not causation".
- The machine is finding a pattern that is ASSOCIATED with the outcome you want to predict.
- There is no implication that this pattern CAUSED the outcome.
Machine Learning Algorithms are a Growth Industry
When you have a reliable prediction algorithm, you have a viable commercial product that can be used to predict that same outcome in different datasets and populations.
FULL DISCLOSURE:
This type of "flight risk" screening has been proven accurate
in industries outside of healthcare. Here is a SHRM article on the topic
19 Questions right off the top of my head ...
- Is your employer surveilling you now with a program like this? Who would you ask to find out?
- Is there a legal requirement for them to inform you of such monitoring?
- If they did tell you the machines will be watching employee physicians in this fashion going forward:
- How would you feel? Is this comforting or distressing?
- Would you stay on or quit and leave?
- If consent was required would you give or withhold it?
- When HR or your leaders come to you and share the information that you have been flagged as a Flight Risk ... how do you imagine that will feel?
- Would you expect your leadership team to fire you or support you to stay?
- Do you think you would trigger a flight risk alert in your current state of job satisfaction and your current energy levels?
- If you are a leader, how would you begin this outreach conversation?
- What resources are available in your organization to support physicians who are struggling and at high risk for quitting?
My core question is what difference does this make?
- To you as an employee physician and your attitude towards your employer?
Is this a good thing?- To you as a leader in the organization?
Does it help you know what to do and who to do it with?- To the turnover rate in the institution?
Will it make a difference to the entity?- To the quality of care and patient experience your physician pool provides?
Will it make a difference to the community?
~~~~~~~~~~~~~~~~~~
PLEASE LEAVE A COMMENT:
Share your thoughts on any of the questions posed above.