According to research, even when a doctor has been trained to use an electronic health record (EHR), finding an answer to a single question can take more than eight minutes on average – this is the result when physicians frequently query a patient’s electronic health record for information that aids in treatment decisions, but the complexity of these records impedes the process.
“Two thousand questions may sound like a lot, but when you look at machine-learning models being trained nowadays, they have so much data, maybe billions of data points. When you train machine-learning models to work in health care settings, you must be creative because there is such a lack of data,” says Eric Lehman, lead author and a Computer Science and Artificial Intelligence Laboratory (CSAIL) graduate student.
The more time doctors must spend navigating an often-clumsy EHR interface, the less time they must interact with patients and provide treatment. US researchers have begun to develop machine-learning models that can automate the process of finding information that physicians require in an EHR. However, training effective models necessitate large datasets of relevant medical questions, which are frequently difficult to obtain due to privacy restrictions.
Existing models struggle to generate authentic questions — those that a human doctor would ask — and are frequently unable to find correct answers. To address this data scarcity, researchers from the Massachusetts Institute of Technology (MIT) collaborated with medical experts to study the questions that physicians have asked when reviewing EHRs. The researchers then created a publicly accessible dataset of over 2,000 clinically relevant questions written by these medical experts.
When they used their dataset to train a machine-learning model to generate clinical questions, they discovered that the model asked high-quality and authentic questions more than 60% of the time and assessed real issues from health experts.
They intend to use this dataset to generate many genuine medical questions, which will then be used to train a machine-learning model that will assist doctors in finding desired information in a patient’s record more efficiently.
The few large datasets of clinical questions the researchers were able to find had a host of issues some were composed of medical questions asked by patients on web forums, which are a far cry from physician questions. Other datasets had questions that were generated using templates, which makes many of the questions implausible.
The MIT researchers collaborated with practising physicians and medical students in their final year of training to create their dataset. They discovered that most questions were centred on the patient’s symptoms, medications, or test findings. Even if these results weren’t a surprise, measuring how many questions there were on each major subject will enable them to create a useful dataset that can be applied in a real-world clinical context.
They also used the publicly available datasets they discovered at the start of this project to train models to recover answers to clinical questions. The trained models were then tested to see if they could find answers to “good” questions posed by human medical experts.
This work is now being applied to the team’s initial goal: developing a model that can automatically answer physicians’ questions in an EHR. They will then use their dataset to train a machine-learning model that can generate thousands or millions of good clinical questions automatically, which will then be used to train a new model for automatic question answering.