Dynamic prediction is an increasingly important task in medical survival analysis, where the goal is to estimate the probability of an event (such as relapse or death) occurring within a given time window, conditional on the patient having survived up to a certain point. Unlike traditional survival analysis that produces static risk estimates, dynamic prediction can update its forecasts as new clinical information becomes available over time.
In this research, we aim to develop multimodal dynamic prediction by integrating multiple data types: tabular data such as lab results, imaging data such as X-rays, and text data such as clinical notes. Since medical data inherently contains missing values and is collected at irregular intervals, we first construct a multimodal imputation method, then build and evaluate prediction models on the imputed data. The goal is to achieve performance that significantly outperforms single-modality approaches.
Genomic analysis is essential for identifying disease-related genes and advancing personalized medicine. This research pursues two complementary approaches. The first leverages ECGPlat, a genomic and proteomic analysis platform developed at DFKI that applies natural language processing techniques to biological sequences. The second approach uses language models designed for DNA and protein sequences — for example, estimating DNA methylation patterns by analyzing the distribution of N-gram patterns within sequences. By combining expertise in computational genomics and AI, we aim to achieve state-of-the-art (SOTA) performance on newly defined analytical tasks.
As AI models grow more complex and opaque, the need for Explainable AI (XAI) becomes critical — particularly in medicine, where physicians must understand and trust AI-generated recommendations before making clinical decisions. Without interpretable explanations, even highly accurate AI systems cannot be responsibly deployed in healthcare.
Building on DFKI's extensive experience with XAI across diverse application domains, this research develops explanation methods tailored to specific medical contexts. We begin with single-modality XAI and extend it to multimodal settings. Evaluation will involve physician surveys and other methods to assess whether the generated explanations are meaningful and convincing to clinical end users.
Large language models (LLMs) and foundation models have the potential to be game changers in healthcare, and their applications are rapidly expanding. Beyond analyzing the vast amounts of textual information stored in electronic health records, LLMs are expected to become a standard tool across the research themes outlined in (1) through (3), including dynamic prediction, genome analysis, and XAI.
This research focuses on effectively integrating LLMs and foundation models into medical data analysis. We conduct systematic model evaluation and selection, prepare the necessary computing infrastructure, and explore LLM-based approaches for tasks such as data imputation and clinical text analysis, drawing on DFKI's deep expertise in natural language processing.