Patrick Leung, CTO of Faro Health, drives the company’s AI-enabled platform, which simplifies and speeds up clinical trial protocol design. Faro Health’s tools enhance efficiency, standardization, and accuracy in trial planning, integrating data-driven insights and streamlined processes to reduce trial risks, costs, and patient burden.
Faro Health empowers clinical research teams to develop optimized, standardized trial protocols faster, advancing innovation in clinical research.
You spent many years building AI at Google. What were some of the most exciting projects you worked on during your time at Google, and how did those experiences shape your approach to AI?
I was on the team that built Google Duplex, a conversational AI system that called restaurants and other businesses on the user’s behalf. This was a top secret project that was full of extremely talented people. The team was fast-moving, constantly trying out new ideas, and there were cool demos of the latest things people were working on every week. It was very inspiring to be on a team like that.
One of the many things I learned on this team is that even when you’re working with the latest AI models, sometimes you still just have to be scrappy to get the user experience and value you want. In order to generate hyper-realistic verbal conversations, the team stitched together recordings interspersed with temporizers like “um” to make the conversation sound more natural. It was so much fun reading what the press had to say about why those “ums” were there after we launched!
Both you and the CEO of Faro come from large tech companies. How has your past experience influenced the development and strategy of Faro?
Several times in my career I’ve built companies that sell various products and services to large companies. Faro too is targeting the world’s largest pharma companies so there is a lot of experience around what it takes to win over and partner with large enterprises that is highly relevant here. Working at Two Sigma, a large algorithmic hedge fund based in New York City, really shaped how I approach data science. They have a rigorous hypothesis-driven process whereby all new ideas go into a research plan and are tested thoroughly. They also have a very well-developed data engineering organization for onboarding new data sets and performing feature engineering. As Faro deepens its AI capabilities to tackle more problems in clinical trial development, this approach will be highly relevant and applicable to what we’re doing.
Faro Health is built around simplifying the complexity of clinical trial design with AI. Coming from a non-clinical background, what was the “aha moment” that led you to understand the specific pain points in protocol design that needed to be addressed?
My first “aha moment” happened when I encountered the concept of “Eroom’s Law”. Eroom isn’t a person, it’s just “Moore” spelt backwards. This tongue-in-cheek name is a reference to the fact that over the past 50 years, inflation adjusted clinical drug development costs and timelines have roughly doubled every 9 years. This flies in the face of the entire information technology revolution, and just boggled my mind. It really sold me on the fact there is an enormous problem to solve here!
As I got deeper into this domain and started understanding the underlying problems more fully, there were many more insights like this. A fundamental and very obvious one is that Word docs are not a good format to design and store highly complex clinical trials! This is a key observation, borne of our CEO Scott’s clinical experience, that Faro was built upon. There is also the observation that over time, trials tend to get more and more complex, as clinical study teams literally copy and paste past protocols, and then add new assessments in order to gather more data. Providing users with as many valuable insights as possible, as early as possible, in the study design process is a key value proposition for Faro.
What role does AI play in Faro’s platform to ensure faster and more accurate clinical trial protocol design? How does Faro’s “AI Co-Author” tool differentiate from other generative AI solutions?
It might sound obvious, but you can’t just ask ChatGPT to generate a clinical trial protocol document. First of all, you need to have highly specific, structured trial information such as the Schedule of Activities represented in detail in order to surface the right information in the highly technical sections of the protocol document. Second, there are many details and specific clauses that need to be present in the documentation for certain types of trials, and a certain style and level of detail that is expected by medical writers and reviewers. At Faro, we built a proprietary protocol evaluation system to ensure the content that the large language model (LLM) was coming up with will meet users’ and regulators’ exacting standards.
As trials for rare diseases and immuno-oncology become more complex, how does Faro ensure that AI can meet these specialized demands without sacrificing accuracy or quality?
A model is only as good as the data it’s trained on. So as the frontier of modern medicine advances, we need to keep pace by training and testing our models with the latest clinical trials. This requires that we continually expand our library of digitized clinical protocols – we are extremely proud of the volume of clinical trial protocols that we have already brought into our data library at Faro, and we’re always prioritizing the growth of this dataset. It also requires us to lean heavily on our in-house team of clinical experts, who constantly evaluate the output of our model and provide any necessary changes to the “evaluation checklists” we use to ensure its accuracy and quality.
Faro’s partnership with Veeva and other leading companies integrates your platform into the wider clinical trial ecosystem. How do these collaborations help streamline the entire trial process, from protocol design to execution?
The heart of a clinical trial is the protocol, which Faro’s Study Designer helps our customers design and optimize. The protocol informs everything downstream about the trial, but traditionally, protocols are designed and stored in Word documents. Thus, one of the big challenges in operationalizing clinical development today is the constant transcription or “translation” of data from the protocol or other document-based sources to other systems or even other documents. As you can imagine, having humans manually translate document-based information into various systems by hand is incredibly inefficient, and introduces many opportunities for errors along the way.
Faro’s vision is a unified platform where the “definition” or elements of a clinical trial can flow from the design system where they are first conceived, downstream to various systems or needed during the operational phase of the trial. When this kind of seamless information flow is in place, there’s a significant opportunity for automation and improved quality, meaning we can dramatically reduce the time and cost to design and implement a clinical trial. Our partnership with Veeva to connect our Study Designer to Veeva Vault EDC is just one step in this direction, with a lot more to come.
What are some of the key challenges AI faces in simplifying clinical trials, and how does Faro overcome them, particularly around ensuring transparency and avoiding issues like bias or hallucination in AI outputs?
There is a much higher bar for clinical trial documents than in most other domains. These documents affect the lives of real people, and thus pass through a highly-exacting regulatory review process. When we first started generating clinical documents using an LLM, it was clear that with off-the-shelf models, the output was nowhere close to meeting expectations. Unsurprisingly, the tone, level of detail, formatting – everything – was way off, and was much more oriented to general-purpose business communications, rather than expert clinical grade documents. For sure hallucination and also straight up omission of necessary details were major challenges. In order to develop a generative AI solution that could meet the high standard for domain specificity and quality that our users expect, we had to spend a lot of time collaborating with clinical experts to devise guidelines and evaluation checklists that ensured our output wasn’t hallucinating or simply omitting key details, and had the right tone. We also needed to provide the capacity for end users to provide their own guidance and corrections to the output, as different customers have differing templates and standards that guide their document authoring process.
There’s also the challenge that the detailed clinical data needed to fully generate the trial protocol documentation may not be readily available, often stored deep in other complex documents such as the investigational brochure. We are looking at using AI to help extract such information and make it available for use in generating clinical protocol document sections.
Looking forward, how do you see AI evolving in the context of clinical trials? What role will Faro play in the digital transformation of this space over the next decade?
As time goes on, AI will help improve and optimize more and more decisions and processes throughout the clinical development process. We will be able to predict key outcomes based on protocol design inputs, like whether the study team can expect enrollment challenges, or whether the study will require an amendment due to operational challenges. With that kind of predictive insight, we will be able to help optimize the downstream operations of the trial, ensuring both sites and patients have the best experience, and that the trial’s likelihood of operational success is as high as possible. In addition to exploring these possibilities, Faro also plans to continue generating a range of different clinical documentation so that all of the filing and paperwork processes of the trial are efficient and much less error-prone. And we foresee a world where AI enables our platform to become a true design partner, engaging clinical scientists in a generative dialog to help them design trials that make the right tradeoffs between patient burden, site burden, time, cost, and complexity.
How does Faro’s focus on patient-centric design impact the efficiency and success of clinical trials, particularly in terms of reducing patient burden and improving study accessibility?
Clinical trials are often caught between the competing needs of collecting more participant data – which means more assessments or tests for the patient – and managing a trial’s operational feasibility, such as its ability to enroll and retain participants. But patient recruitment and retention are some of the most significant challenges to the successful completion of a clinical trial today – by some estimates, as many as 20-30% of patients who elect to participate in a clinical trial will ultimately drop out due to the burden of participation, including frequent visits, invasive procedures and complex protocols. Although clinical research teams are aware of the impact of high burden trials on patients, actually doing anything concrete to reduce burden can be hard in practice. We believe one of the barriers to reducing patient burden is often the inability to readily quantify it – it’s hard to measure the impact to patients when your design is in a Word document or a pdf.
Using Faro’s Study Designer, clinical development teams can get real-time insights into the impact of their specific protocol on patient burden during the protocol planning process itself. By structuring trials and providing analytical insights into their cost, patient burden, complexity early during the trials’ design stage, Faro provides clinical research teams with a very effective way to optimize their trial designs by balancing these factors against scientific needs to collect more data. Our customers love the fact we give them visibility into patient burden and related metrics at a point in development where changes are easy to make, and they can make informed tradeoffs where necessary. Ultimately, we have seen our customers save thousands of hours of collective patient time, which we know will have an immediate positive impact for study participants, while also helping ensure clinical trials can both initiate and complete on time.
What advice would you give to startups or companies looking to integrate AI into their clinical trial processes, based on your experiences at both Google and Faro?
Here are the main takeaways I’d offer so far from our experience applying AI to this domain:
- Divide and evaluate your AI prompts. Large language models like GPT are not designed to output clinical grade documentation. So if you’re planning to use gen AI to automate clinical trial document authoring, you need to have an evaluation framework that ensures the generated output is accurate, complete, has the right level of detail and tone, and so on. This requires a lot of careful testing of the model guided by clinical experts.
- Use a structured representation of a trial. There is no way you can generate the required data analytics in order to design an optimal clinical trial without a structured repository. Many companies today use Word docs – not even Excel! – to model clinical trials. This must be done with a structured domain model that accurately represents the complexity of a trial – its schema, objectives and endpoints, schedule of assessments, and so on. This requires a lot of input and feedback from clinical experts.
- Clinical experts are crucial for quality. As seen in the previous two points, having clinical experts directly involved in the design and testing of any AI based clinical development system is absolutely critical. This is much more so than any other domain I’ve worked in, simply because the knowledge required is so specialized, detailed, and pervades any product you attempt to build in this space.
We are constantly trying new things and regularly share our findings to our blog to help companies navigate this space.
Thank you for the great interview, readers who wish to learn more should visit Faro Health.
Credit: Source link
