Watson for Clinical Trial Matching
Clinical trials are conducted to prove that new treatments are effective and safe in order to get regulatory approval. For hospitals, conducting clinical trials can be lucrative and benefit their patients, but it can be difficult to enroll enough patients to fulfill the requirements of the trial. In fact, nearly 1 in 4 clinical trials fail because they cannot enroll enough patients.
In 2015, IBM Watson Health collaborated with the Mayo Clinic to develop Clinical Trial Matching (CTM), a platform to help identify and screen patients for clinical trials. However, CTM had suffered from low usage, and our clients complained that the platform was difficult to use.
In 2017, I worked with a team of designers to reimagine the clinical trial experience and realign the product with our user’s ideal workflow.
I was the UX Designer responsible for the interaction design, information architecture, and content hierarchy of CTM and worked alongside a design lead, design researcher, and visual designer.
Designing for the right user
When I began working on this product, we had large clients, but few active users. Our team discovered that although CTM was designed wtih an Oncologist in mind, Oncologists were not the ones responsible for enrolling patients into clinical trials. The main challenge of the redesign process our team executed was to redesign the application to suit the needs of Clinical Trial Coordinators, who work to recruit patients for clinical trials.
Identifying the user
We spoke to dozens of users across medical institutions to help identify the role of our user in their organization. Before our design team came on board, Watson for Clinical Trial Matching focused on helping physicians identify patients for clinical trials. However, through interviews and evaluative testing, we discovered Oncologists enroll only 20% of study participants. Instead, it was the role of Clinical Trial Coordinators at the institution to identify and recruit patients for each trial. This led us to shift our application from being patient-centric to trial-centric, and we made the case that our user was the Clinical Trial Coordinator.
Our primary user is the Clinical Trial Coordinator, who works for the hospital or medical institution where the trial is conducted. The Trial Coordinator’s role is to scour the institution's existing patients to identify and enroll those who would be a good fit for the trial based on pre-defined criteria, such as age, gender, and pre-existing conditions.
Our secondary user would be the Research Manager, who oversees the work of the Trial Coordinators, liases with the pharmaceutical company that is sponsoring the trial, works with the clinical working group to assess trial feasibility and logistics, secures grant funding for studies, and ensures the team is prepared for audits. The Research Manager may also take on the responsibilities of Coordinators when the site is short-staffed.
Another secondary user would be a physician at the institution who could notify a Trial Coordinator if they are interested in enrolling a patient into a clinical trial because they think it is their patient’s best option. Physicians also would need to view and update their patients’ information periodically so that trial coordinators are using the most accurate data to enroll patients.
Our end user is the patient, who wants to receive the most effective and safest treatment option available to them. Hasn’t responded well to FDA-approved treatments, so experimental treatments are often their last hope.
Why do Trial Coordinators have trouble recruiting the right patients?
Lack of eligible patients
In some instances, trials may be recruiting for patients with a rare combination of conditions.
Lack of patient interest
Because the treatments are experimental, few physicians and patients want to try the treatment if they can find an approved drug.
Onerous screening process
Depending on the trial criteria, patients will have to submit to a battery of tests to determine whether they meet all the criteria for a trial subject. Complicated biopsies and lab test stress out patients and care providers, and there are strict deadlines for when these test results must be submitted. For Trial Coordinators, in order to avoid missing an eligible patient, they would have to meticulously review the records of every single patient and stay on top of an overwhelming schedule of lab tests for patients who are being enrolled.
Rethinking the user flow
In the earlier design, the landing page displayed a long list of Clinical Trials intended for oncologists to browse and select for their patients. In our new design, we wanted to reorient the experience around how the Trial Coordinator manages enrollment for their trials.
The new user flow centered around a landing page where Trial Coordinators could see actionable updates about their assigned trials, linking to a full project management view of all the patients they were trying to enroll and what stage they were in.
The mindset of the Trial Coordinator is task and deadline oriented, and the first question on their minds is always “What do I need to do next to enroll patients on my trials?”
We introduced a landing page called “My Trials” for Trial Coordinators to save the trials they were assigned to. Rather than being just a list of trials, the My Trials page tells Coordinators what they need to do next and reminds them of newly identified and excluded patients to focus on. By clicking these links, they are able to jump to those patients in tracker right away.
In early iterations of this page, we provided actionable prompts to change the patient's status directly from this page. We made the assumption that coordinators would want to review Watson's recommendations and accept them right away, but from speaking with users, we came to realize that they would always want to first review that patient's case before changing their status. As a result, in the final design we prompted the user to review the patient's profile instead.
The Tracker Board
Trial Coordinators told us they needed a full view of all the patients they were enrolling. Inspired by the color coded calendars they used to manage their enrollments, we brainstormed ways we could translate this visual representation of their enrollments into digital form.
We focused on displaying where patients were in the stages of enrollment in order to help Trial Coordinators gain control and visibility in the stringent multi-step process.
After researching different ways other products helped users manage their workflow, we chose to use a Kanban board because it would enable us to clearly show the linear stages of enrollment, and allowed the user to see how many patients they had in each stage, while moving patients between these different stages.
In our early wireframes, we defined how a user would navigate seamlessly between their trials, the tracker board showing all patients, and a deep dive into each patient's individual profile.
We also explored many options for the visual treatment and features of the Kanban cards. Ultimately we decided to remove the information about the number of criteria a patient has met in this view, as a coordinator would need to view the patient's entire profile to make a decision on their enrollment status. Thus, showing the information would not be helpful in this summary view. We also decided against showing a checklist of items within a card, opting to show only the most urgent Watson recommendations to focus the coordinator on the most actionable tasks.
One consideration we made here was how many patients would typically show up in this Kanban board, and when we should start truncating the patients. Based on our research, there are usually 30-40 patients being actively enrolled at once, although the Identified number could be higher. We also decided to show patients with updates or upcoming deadlines higher on the list and truncate beyond the first 15. Clicking view all would take the user to a full list of patients in that enrollment stage.
Working with Watson
At Watson Health, we strived to make each interaction with Watson an exchange with a helpful colleague. Watson can help save users time and surface relevant data they weren’t aware of, but at this time, he cannot replace their medical expertise and judgment. Therefore, Watson is useful for prescreening patients and providing Trial Coordinators with relevant information as they decide which patients to enroll.
By bookmarking clinical trials, Coordinators let Watson know that they want to prescreen patients for these trials. Watson will access the medical records of any patients with upcoming appointments at the hospital and compare them to the trial criteria using Natural Language Processing.
Watson will then sort patients into Identified and Excluded groups, which are shown as updates on the My Trials page. This can save Trial Coordinator hours of work ruling out each individual patient and ensures that no eligible patients are missed.
Based on Watson’s findings in My Trials, the user can decide whether to change the patient’s enrollment status in the Tracker. Watson also notifies the user when there is a change in the patient’s status (such as when a new lab result has come in).
By clicking "View All" the user can pull up a list of all the Identified patients with relevant details in a separate screen.
The user can verify the eligibility of a patient by opening up their medical records and comparing them to the trial criteria. Watson has prepopulated the patient’s lab values from the medical records, but if the Trial Coordinator or an oncologist has any new information, they can manually input that information and ask Watson to update that patient’s eligibility status.
If the patient is eligible, the Trial Coordinator will work with the patient’s physician to obtain physician and patient consent, as well as new lab tests to enroll this patient. Watson automatically detects changes to the patient's records and will suggest moving this patient in Tracker when their eligibility status changes.
The Oncologist’s experience
Watson recommends trials for patients based on their medical records. If an oncologist thinks their patient is right for a trial, they can mark this trial as a preferred treatment option, and the Trial Coordinator will see the Physician Preferred label on the patient. While Oncologists may not have time to review all the details of the trial on CTM, now they have greater visibility into where their patients are in the process of enrollment.
By streamlining the enrollment process, it’s likelier patients will be enrolled in clinical trials that are right for them. There’s a better chance that they will be identified for the right trials and that they won’t miss criteria deadlines because Watson tracks new lab results and any changes to their medical records.
Positive Clinical Outcomes
Mayo Clinic experienced an 80% increase in Breast Cancer clinical trial enrollment when using the redesigned Watson for Clinical Trial Matching platform.
Clinical Trial Matching also significantly reduced the time to screen a patient for clinical trial matches, according to Dr Kyu Rhee, chief health officer of Watson Health.
When Mayo Clinic first implemented Watson for Clinical Trial Matching in July 2016, it had only been trained for breast cancer, but since then the system has evolved to support colorectal, lung, and gastrointestinal cancers. Additional training is underway to train the system on more cancers in the near future.
Mayo Clinic and IBM continue to work together to expand the use of CTM in oncology practice.
"This has enabled all patients to be screened for all available clinical trial opportunities…The speed and accuracy of Watson and the team of screening coordinators allow our physicians to efficiently develop treatment plans for patients that reflect the full range of options available to support their care.”
Tufia Haddad, MD
Mayo Clinic oncologist and ctm physician leader