This article was written by Sergii Gorpynich, co-Founder and CTO at Star, co-written by Perry Simpson, Managing Director of Star, and was written, edited and published in alignment with our transparent Emerj sponsored content guidelines. Learn more about reaching our AI-focused executive audience on our Emerj advertising page.
As AI and data science make their way into more and more healthcare processes, from patient care to diagnosis and prognosis, healthcare firms are struggling to realize AI’s benefits and thereby master the use of their own data.
While cataloging the state of a healthcare firm’s data ecosystem can be challenging, data issues go beyond identifying the state of existing data assets. Business and technology leadership, therefore, must be able to identify the right, high-impact use cases for data science and AI within the organization.
Many leadership teams have an initial idea of how they’d like to use AI, they’re nevertheless unsure of how to execute the initiative end-to-end—from idea to production-grade system.
In this article, we break down the stages that are critical to bringing an AI project from brainstorm to launch. When it comes to unlocking healthcare data and selecting high-value AI initiatives, Star’s data science teams follow three distinct phases:
- Data Research: This exploratory data analysis state involves a multidisciplinary dialogue between business stakeholders, subject-matter experts, and AI/data science experts to assess data assets and determine high-value applications
- AI Model Prototyping and Testing: Initial experiments and applicable machine learning modeling to find the best potential solution
- AI Implementation: Integrating an AI initiative or product with core systems, teams and processes
We’ll focus mostly on the first two phases in the process, breaking down their component parts in a framework that healthcare leaders should find useful and be able to use immediately.
For more on adopting AI in the healthcare industry, download the Emerj-Star white paper, Executive Guide to AI Adoption in Healthcare.
Data Research
Research Process and Objectives
The first step lies with business stakeholders determining what they want to achieve and what data assets they have access to.
This involves meeting with business leadership, subject matter experts, and experienced data scientists to assess the current situation and determine opportunity areas. The group works together to understand the business’s goals, its ecosystem, and existing data assets.
With regard to the data assets, teams should ask: are there meaningful business insights that we could derive from these data? Is there a way we could apply these insights to directly impact our business?
Company leaders are aware of AI and have ideas on the outcomes they want to create. Examples include determining the incidence of hospital acquired infections, mastering EHR management to relieve laborious data entry and interpretation, optimizing appointment and staffing, etc.
Subject-matter experts may also have their own ideas regarding implementing machine learning to drive value, and data scientists will be able to assess the technical viability of their ideas.
This mix of business leadership, subject matter expertise and data science is critical in the data research phase.
For instance, a healthcare company might hire a team of data scientists to leverage machine learning, but these data scientists aren’t going to necessarily know anything about healthcare data, let alone the specific business model, patient profiles, or data ecosystem of their company.
Data scientists need business and process context in order to know what data means and how data have historically been valuable in solving business problems.
A data scientist with access to appointment data, for instance, may not realize the different staff and personnel requirements for handling dialysis treatments versus chemotherapy. The data scientist won’t know the operational leaders’ “rules of thumb” for how appointments are booked and managed today and why.
Similarly, a business leader may see value in using medical imagery data to train a computer vision system to better diagnose lung cancer, but the same leader may have no idea how much data would be required to achieve such a goal or the lengthy extent of the data labeling needed to train a machine learning model.
Bringing these groups and perspectives together generates ideas that are both viable (technically) and valuable (economically or to the end user).
Data research is a collaborative exploratory data analysis process which may take several days to 2-4 weeks of team workshops. Data scientists would come out of this phase with a good understanding of the types of data available and where to find them, as well as whether or not the data need to be harmonized or if more data need to be collected first.
In addition, insights from subject-matter experts and stakeholders (radiologists, nurses, customer service managers, etc.) should leave them with a strong sense of the problem the business wants to solve. With this in mind, data scientists can execute on their plans and start prototyping a model.
The Importance of Combined Expertise
In the data research phase, subject-matter experts should help generate ideas and provide context for data scientists. If a pilot or prototype project is chosen within the subject-matter expert’s domain (for example: radiology, customer service, EHR, etc.), then that subject-matter expert and/or their team should have consistent communication with the data scientists.
Subject-matter experts are essential to this stage of the AI-building process because they steer data scientists in the right direction. A lead data scientist might think of a dozen types of data they could use to build a machine learning model for a particular use-case.
A subject-matter expert can then provide focus by confirming more than half of those data types are sparse and rare, are poorly organized or collected, or that using them could violate regulations.
Another important perspective subject-matter experts bring to the table is the ability to find value in the data.
Ultimately, what we’re trying to search for in the data is some meaningful business insights. So we’re not looking at the data from a technical perspective…That’s…a secondary question. The first question is: Might we be able to extract any meaningful insights which will be important for the business?
Subject-matter experts can help answer these critical questions, determining the most fruitful areas on which to focus the company’s AI efforts.
Nevertheless, the reliance goes both ways, and subject-matter experts need data scientists to ground their ideas in what is technically feasible. A radiologist may want to use a computer vision system to detect lung cancer and apply the same application to a variety of other oncology diagnostic tasks.
A data scientist could then determine if the data for these other diagnostic tasks are sufficient to train a model or if the data are similar enough to transfer from one diagnostic task to another. A data scientist who understands the business context, therefore, will be able to determine the idea’s viability.
AI Model Prototyping and Testing
Why the Prototyping Phase is Necessary
Once a project is determined and data can be accessed, a technical team will need to get to work on a model prototype in order to determine whether a machine learning model can deliver the results business leadership is looking for.
For example:
- A system to predict optimal E&M codes. Data scientists and a project team may be tasked with applying a logistic model to past EHR and billing data to better predict an optimal evaluation & management codes. Leveraging NLP and a classification algorithm to medical records brings a new degree of automation to what is otherwise a very manual and laborious process. Finding the right combination of data features and models would involve technical iteration and open dialogue between data scientists and subject matter experts.
- Detecting lung cancer using machine vision. Data scientists could work with radiologists and oncologists to determine the criteria that lead to an X-ray diagnosis, and to ensure that thousands of previous lung cancer X-rays were properly labeled and highlighted in order to train a computer vision model to detect lung cancer in new X-ray images.
- Leveraging AI to predict the spread of illness in real time. Using millions of anonymous data inputs collected from a smart connected thermometer, an AI model can be built to test the efficacy of triaging illnesses based on severity, contagion and duration. Further, ensuring these data are labeled with geo-location coordinates, the model can test its ability to predict the spread of disease, ultimately enabling officials to take action and minimize its impact on an entire population.
Any AI-enabled healthcare capability would involve a period of data collection, model training, and iterative testing.
Like the exploratory data analysis process, this phase involves ongoing dialogue between the data scientists and subject-matter experts to help train the model and to ensure its outputs (optimizing E&M coding, detecting lung cancer, or predicting illness across populations) are accurate and useful.
While the C-suite may be involved in brainstorming and determining an initial project, they are less likely to be involved in the prototyping phase, where communication between data scientists and subject-matter experts becomes most important.
Outside ongoing dialogue with subject-matter experts, data scientists will spend their time on technical tasks. They’ll clean and harmonize the data (if need be) and work on training algorithms to deliver the desired result. They’ll determine which features in the data to use and which algorithms or statistical methods they’ll apply.
Goals of the Model Prototyping and Testing Phase
A Model Prototyping and Testing phase exists in order to explore and prove the value of an AI application before bringing it into production. As such, project managers and senior leaders usually aim to reach specific benchmarks and objectives with their prototypes.
The Viability of the Model with Limited Data
One of the goals of this prototyping phase is to test the machine learning hypothesis on a limited dataset. For example, a computer vision model built to detect the presence of cancer from images of patient CAT scans will first need to be tested on a sizable portion of the overall dataset (i.e. the healthcare network’s entire archive of CAT scans).
If the algorithm correctly detects cancer within these CAT scans within a percentage of time likely determined by stakeholders, the prototype may be considered a success. Determining these specific criteria for success upfront is critical.
The Fundamental User Interface
Another goal during this stage of the AI building process involves determining how data scientists are to build the machine learning product’s user interface. Subject-matter experts will need to know who is going to be using the product. Doctors? Surgeons? Patients?
The answer will greatly affect how the product is built to allow for the greatest efficiency and, in some cases, the least risk. An effective model by itself won’t drive business value. With a useable interface, stakeholders can begin to see how their AI initiative could go from experimentation to production and serve the needs of real users.
The Economic Viability of the Project or Application
If business leadership is happy with the output of the models, they’ll then assess the resources required to bring the model into production. Determinative questions might include:
- What will it cost to train staff to use this new AI-based solution within our workflows?
- What are the ongoing costs of acquiring, cleaning, and labeling data in order to keep this system working?
- What other IT systems will need to be changed, replaced, or integrated with this new AI-based system?
- What kind of results would we need to see from this AI system before approving the resources required to run it?
AI Implementation
Once the model is tested, an effective user interface is built and the data are made accessible, the system may be ready for deployment. Implementation occurs once a model has been validated, budgets have been approved, and the leadership team believes that data assets are sufficient to deliver the business result.
If a model shows promise, and business leadership believes that its value would be worth the implementation investment, then a technical team would be tasked with delivering a scalable AI initiative into production.
Healthcare companies looking to adopt machine learning need to understand that AI is an investment in their firm’s future. The lessons learned in applying initial AI projects will often bolster a company’s ability to deploy AI applications and tools going forward.
The multi-disciplinary meetings, changes to data infrastructure, and training of unique models will all be transferrable skills for other AI projects, preparing a company to be more capable of developing other AI initiatives. Beyond potential improvements to efficiencies or patient outcomes, these are additional benefits from deploying a first AI process.
Header image credit: Mashable
