High-quality synthetic medical images can enlarge training datasets in different deep learning-based applications. Recently, diffusion-based methods for image synthesis have outperformed GAN-based methods, even for medical images. Unfortunately, using diffusion models is costly in terms of training time and computational resources. We propose a two-stage method that combines diffusion models and GANs to tackle this problem. First, we use diffusion models or GANs to generate low-resolution images. Then, we use a GAN-based super-resolution model to interpolate high-resolution images from these low-resolution images. Experimental results on synthetic breast CT slices show that the proposed framework is more efficient and performs better than state-of-the-art methods that generate the images in a single step. The proposed methods will be available at https://github.com/xiaoerlaigeid/Image-Frequency-Score.git.

Objective. Glioblastoma (GBM) prognosis remains poor despite progress in radiotherapy and imaging techniques. Tumor recurrence has been attributed to the widespread tumor invasion of normal tissue. Since the complete extension of invasion is undetectable on imaging, it is not deliberately treated. To improve the treatment outcome, models have been developed to predict tumor invasion based standard imaging data. This study aimed to investigate whether a tumor invasion model, together with the predicted number of surviving cells after radiotherapy, could predict tumor progression post-treatment. Approach. A tumor invasion model was applied to 56 cases of GBMs treated with radiotherapy. The invasion was quantified as the volume encompassed by the 100 cells mm−3 isocontour (V100). A new metric, cell-volume-product, was defined as the product of the volume with cell density greater than a threshold value (in cells mm−3), and the number of surviving cells within that volume, post-treatment. Tumor progression was assessed at 20 ± 10 d and 90 ± 20 d after treatment. Correlations between the disease progression and the gross tumor volume (GTV), V100, and cell-volume-product, were determined using receiver operating characteristic curves. Main results. For the early follow-up time, the correlation between GTV and tumor progression was not statistically significant (p = 0.684). However, statistically significant correlations with progression were found between V100 and cell-volume-product with a cell threshold of 10−6 cells mm−3 with areas-under-the-curve of 0.69 (p = 0.023) and 0.66 (p = 0.045), respectively. No significant correlations were found for the late follow-up time. Significance. Modeling tumor spread otherwise undetectable on conventional imaging, as well as radiobiological model predictions of cell survival after treatment, may provide useful information regarding the likelihood of tumor progression at an early follow-up time point, which could potentially lead to improved treatment decisions for patients with GBMs.

Purpose: This study aimed at applying a mathematical framework for the prediction of high-grade gliomas (HGGs) cell invasion into normal tissues for guiding the clinical target delineation, and at investigating the possibility of using tumor infiltration maps for patient overall survival (OS) prediction. Material & methods: A model describing tumor infiltration into normal tissue was applied to 93 HGG cases. Tumor infiltration maps and corresponding isocontours with different cell densities were produced. ROC curves were used to seek correlations between the patient OS and the volume encompassed by a particular isocontour. Area-Under-the-Curve (AUC) values were used to determine the isocontour having the highest predictive ability. The optimal cut-off volume, having the highest sensitivity and specificity, for each isocontour was used to divide the patients in two groups for a Kaplan-Meier survival analysis. Results: The highest AUC value was obtained for the isocontour of cell densities 1000 cells/mm3 and 2000 cells/mm3, equal to 0.77 (p < 0.05). Correlation with the GTV yielded an AUC of 0.73 (p < 0.05). The Kaplan-Meier survival analysis using the 1000 cells/mm3 isocontour and the ROC optimal cut-off volume for patient group selection rendered a hazard ratio (HR) of 2.7 (p < 0.05), while the GTV rendered a HR = 1.6 (p < 0.05). Conclusion: The simulated tumor cell invasion is a stronger predictor of overall survival than the segmented GTV, indicating the importance of using mathematical models for cell invasion to assist in the definition of the target for HGG patients.

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally-invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow.

This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis.

The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head-neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy.

Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power.

Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra-dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses.

In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis.

Cancer är en global hälsoutmaning som uppskattas ansvara för cirka 10 miljoner dödsfall i hela världen, bara under året 2020. Framsteg inom medicinsk bildtagning och hårdvaruutveckling de senaste tre decennierna har banat vägen för moderna medicinska bildgivande system vars upplösningsförmåga tillåter att fånga information om tumörers anatomi, fysiologi, funktion samt metabolism. Medicinsk bildanalys har därför fått en mer betydelserik roll i klinikers dagliga rutiner inom onkologin, för bland annat screening, diagnostik, uppföljning av behandling samt icke-invasiv utvärdering av sjukdomsprognoser. Sjukvårdens behov av medicinska bilder har lett till att det nu på sjukhusen finns en enorm mängd medicinska bilder på alla moderna sjukhus. Med hänsyn till den viktiga roll medicinsk bilddata spelar i dagens sjukvård, samt den mängd manuellt arbete som behöver göras för att analysera den mängd data som genereras varje dag, så har utvecklingen av digitala verktyg för att för att automatiskt eller semi-automatiskt analysera  bilddatan alltid haft stort intresse. Därför har en rad maskininlärningsverktyg utvecklats för analys av onkologisk data, för att gripa sig an läkares repetitiva vardagssysslor.

Den här avhandlingen syftar att bidra till fältet “onkologisk bildanalys” genom att föreslå nya sätt att kvantifiera tumörers egenskaper från medicinsk bilddata. Specifikt, är denna avhandling baserad på sex artiklar där de första två har fokus att presentera nya metoder för segmentering av tumörer, och de resterande fyra ämnar att utveckla kvantitativa biomarkörer för cancerdiagnostik och prognos.

Huvudsyftet för “Studie I” har varit att utveckla en djupinlärnings-pipeline vars syfte är att fånga lungpatalogiers anatomier (inklusive lungtumörer) samt integrera detta med djupa neurala nätverk för segmentering för att nyttja det första nätverkets utfall för att förbättra segmenteringskvalitén. Den föreslagna pipelinen testades på flertalet dataset och numeriska analyser visar en överlägsna resultat för den föreslagna “prior-medvetna” djupinlärningsmetoden. “Studie II” ämnar att ta sig an ett viktig problem som övervakade segmenteringsmetoder ställs inför: ett beroende av enorma annoterade dataset. I denna studie föreslås en icke-övervakad segmenteringsmetod som baseras på konceptet “ifyllnad” (“inpainting”) för att segmentera tumörer i områdena: lungor samt huvud och hals i bilder från olika modaliteter. Den föreslagna metoden lyckas bättre än en familj väletablerade icke-oövervakade segmenteringsmodeller.

“Studie III” och “Studie IV” försöker automatiskt diskriminera benigna lungtumörer från maligna tumörer genom att analysera bilder från LDCT (lågdos-CT). I “Studie III“ föreslås ett djupt neuralt nätverk för klassificering vars grafstruktur tillåter lokal analys av tumörens inbördes heterogeniteter samt en helhetsbild från global kontextuell information. “Studie IV” försöker utvärdera noggrant utvalda metoder som grundar sig på att extrahera anatomiska särdrag från medicinska bilder. I studien jämförs konventionella “radiomics”-metoder med särdrag från neurala nätverk samt en kombination av båda på samma dataset. Resultat från studien visar att en kombination av särdrag från djupa neurala nätverk samt “radiomics” kan ge bättre resultat i klassificeringsproblemet.

“Studie V” har fokus på tidig bedömning av lungtumörers respons på behandling genom att utveckla ett set nya fysiologisk observerbara särdrag. Den presenterade metoden har använts för att kvantifiera förändringar i tumörers karaktär i PET-CT-undersökningar för att predicera patienters prognos två år efter senaste behandling. Metoden jämförts mot konventionella “radiomics” och utvärderingen visar att den föreslagna metoden ger förbättrade resultat. Till skilnad från “Studie V”, som fokuserar på att lösa ett binärt klassificeringsproblem, så försöker “Studie VI” predicera överlevnadsgraden hos patienter med lung- samt huvud och hals-cancer genom att undersöka neurala nätverk med sfäriska faltningsoperationer. Metoden jämförs mot, bland annat, “radiomics” och visar liknande resultat för analys på samma dataset, men bättre resultat för analys på olika dataset.

Sammanfattningsvis så utnyttjar de sex studierna olika medicinska bildgivande system samt en mängd olika bildbehandling- och maskininlärningstekniker för att utveckla verktyg för att kvantifierar tumörers egenskaper, som kan underlätta fastställande av diagnos och prognos.

Purpose: High-grade glioma (HGG) is a common form of malignant primary brain cancer with poor prognosis. The diffusive nature of HGGs implies that tumor cell invasion of normal tissue extends several centimeters away from the visible gross tumor volume (GTV). The standard methodology for clinical volume target (CTV) delineation is to apply a 2-to 3-cm margin around the GTV. However, tumor recurrence is extremely frequent. The purpose of this paper was to introduce a framework and computational model for the prediction of normal tissue HGG cell invasion and to investigate the agreement of the conventional CTV delineation with respect to the predicted tumor invasion. Methods and Materials: A model for HGG cell diffusion and proliferation was implemented and used to assess the tumor invasion patterns for 112 cases of HGGs. Normal brain structures and tissues as well as the GTVs visible on diagnostic images were delineated using automated methods. The volumes encompassed by different tumor cell concentration isolines calculated using the model for invasion were compared with the conventionally delineated CTVs, and the differences were analyzed. The 3-dimensional-Hausdorff distance between the CTV and the volumes encompassed by various isolines was also calculated. Results: In 50% of cases, the CTV failed to encompass regions containing tumor cell concentrations of 614 cells/mm3 or greater. In 84% of cases, the lowest cell concentration completely encompassed by the CTV was & GE;1 cell/mm3. In the remaining 16%, the CTV overextended into normal tissue. The Hausdorff distance was on average comparable to the CTV margin. Conclusions: The standard methodology for CTV delineation appears to be inconsistent with HGG invasion patterns in terms of size and shape. Tumor invasion modeling could therefore be useful in assisting in the CTV delineation for HGGs.

Segmentation of lung pathology in Computed Tomography (CT) images is of great importance for lung disease screening. However, the presence of different types of lung pathologies with a wide range of heterogeneities in size, shape, location, and texture, on one side, and their visual similarity with respect to surrounding tissues, on the other side, make it challenging to perform reliable automatic lesion seg-mentation. To leverage segmentation performance, we propose a deep learning framework comprising a Normal Appearance Autoencoder (NAA) model to learn the distribution of healthy lung regions and re-construct pathology-free images from the corresponding pathological inputs by replacing the pathological regions with the characteristics of healthy tissues. Detected regions that represent prior information re-garding the shape and location of pathologies are then integrated into a segmentation network to guide the attention of the model into more meaningful delineations. The proposed pipeline was tested on three types of lung pathologies, including pulmonary nodules, Non-Small Cell Lung Cancer (NSCLC), and Covid-19 lesion on five comprehensive datasets. The results show the superiority of the proposed prior model, which outperformed the baseline segmentation models in all the cases with significant margins. On av-erage, adding the prior model improved the Dice coefficient for the segmentation of lung nodules by 0.038, NSCLCs by 0.101, and Covid-19 lesions by 0.041. We conclude that the proposed NAA model pro-duces reliable prior knowledge regarding the lung pathologies, and integrating such knowledge into a prior segmentation network leads to more accurate delineations.

ObjectiveSurvival Rate Prediction (SRP) is a valuable tool to assist in the clinical diagnosis and treatment planning of lung cancer patients. In recent years, deep learning (DL) based methods have shown great potential in medical image processing in general and SRP in particular. This study proposes a fully-automated method for SRP from computed tomography (CT) images, which combines an automatic segmentation of the tumor and a DL-based method for extracting rotational-invariant features. MethodsIn the first stage, the tumor is segmented from the CT image of the lungs. Here, we use a deep-learning-based method that entails a variational autoencoder to provide more information to a U-Net segmentation model. Next, the 3D volumetric image of the tumor is projected onto 2D spherical maps. These spherical maps serve as inputs for a spherical convolutional neural network that approximates the log risk for a generalized Cox proportional hazard model. ResultsThe proposed method is compared with 17 baseline methods that combine different feature sets and prediction models using three publicly-available datasets: Lung1 (n=422), Lung3 (n=89), and H&N1 (n=136). We observed comparable C-index scores compared to the best-performing baseline methods in a 5-fold cross-validation on Lung1 (0.59 +/- 0.03 vs. 0.62 +/- 0.04). In comparison, it slightly outperforms all methods in inter-data set evaluation (0.64 vs. 0.63). The best-performing method from the first experiment reduced its performance to 0.61 and 0.62 for Lung3 and H&N1, respectively. DiscussionThe experiments suggest that the performance of spherical features is comparable with previous approaches, but they generalize better when applied to unseen datasets. That might imply that orientation-independent shape features are relevant for SRP. The performance of the proposed method was very similar, using manual and automatic segmentation methods. This makes the proposed model useful in cases where expert annotations are not available or difficult to obtain.