kth.sePublications
Planned maintenance
A system upgrade is planned for 10/12-2024, at 12:00-13:00. During this time DiVA will be unavailable.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Geometric Deep Learning for Medical Image Processing Problems
KTH, School of Engineering Sciences in Chemistry, Biotechnology and Health (CBH), Biomedical Engineering and Health Systems, Medical Imaging. Karolinska Institute.ORCID iD: 0000-0003-3502-7441
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Sustainable development
SDG 3: Good Health and Well-Being, SDG 10: Reduced inequalities
Abstract [en]

This thesis consists of four studies, each suggesting incorporating geometric deep-learning methods in medical image-processing pipelines.

The rationale here is that a) conventional Deep learning (DL) models for medical imaging depend highly on the quality and quantity of training data, and b) medical images commonly represent structures with intrinsic geometrical properties that standard DL models do not necessarily respect. 

The first study focuses on predicting stiffness tensors from micro-CT trabecular bone scans. This prediction task requires learning from the complex structures of trabecular bone with limited data available. Our proposed model uses a spherical convolutional neural network (SphCNN) for this purpose. The second study investigates lung cancer survival rate prediction based on image-based features. We evaluate a proposed SphCNN-based pipeline using CT images of non-small cell lung cancer. The third study centres on the stability of the streamline tractography algorithm under arbitrary 3D rotations. We propose integrating an SE(3)-equivariant transformer model into the tractography framework to preserve rotational equivariance. The fourth study evaluates a structural connectivity pipeline in combination with tractography filtering, subsequent classification and analysis. The pipeline is evaluated based on its ability to identify group differences in brain connectivity related to Parkinson's disease. 

Abstract [sv]

I denna avhandling presenteras fyra studier som tillsammans utforskar en integration av geometriska djupinlärningsmetoder i bildbehandling för medicinska tillämpningar.

Motivationen till detta är att a) konventionella djupinlärningsmodeller för medicinsk avbildning i hög grad är beroende av träningsdatans mängd och kvalitet, samt b) medicinska bilder ofta avbildar strukturer med inneboende geometriska egenskaper som traditionella djupinlärningsmodeller inte nödvändigtvis tar hänsyn till.

Den första studien fokuserar på att förutsäga styvhetstensorer baserat på mikro-CT-bilder av trabekulärt ben. Denna förutsägelse kräver inlärning från de komplexa strukturerna i trabekulärt ben med begränsad tillgång på data. Vår föreslagna modell använder ett sfäriskt faltningseuronnät  (SphCNN) för detta ändamål. Den andra studien undersöker överlevnadsprognoser vid lungcancer utifrån bildbaserade egenskaper. Vi utvärderar en föreslagen pipeline, baserad på SphCNN, med CT-bilder av icke-småcellig lungcancer. Den tredje studien fokuserar på stabiliteten hos en traktografialgoritm för riktade fibrer under godtyckliga tredimensionella rotationer. Vi föreslår att integrera en SE(3)-ekvivariant transformermodell i traktografins ramverk för att bevara rotations-ekvivariansen. Den fjärde studien utvärderar en pipeline för strukturell konnektivitet i kombination med traktogramfiltrering, klassificering och analys. Pipelinen utvärderas utifrån dess förmåga att identifiera skillnader i hjärnans konnektivitet kopplat till Parkinsons sjukdom.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. , p. 114
Series
TRITA-CBH-FOU ; 2024:48
Keywords [en]
Geometric Deep Learning, Medical Image Processing, Diffusion Brain MRI, Spherical Convolutional Neural Networks, Structural Connectivity, Bone Stiffness Tensor Prediction, Lung Cancer Survival Rate Prediction, Tractography
Keywords [sv]
Geometriskt djupinlärning, Medicinsk bildbehandling, Diffusions hjärn-MRI, Sfäriska konvolutionella neurala nätverk, Strukturell konnektivitet, Styvhetstensorprediktion för ben, Lungcancersöverlevnadsprognos, Traktografi
National Category
Medical Image Processing
Research subject
Medical Technology
Identifiers
URN: urn:nbn:se:kth:diva-355899ISBN: 978-91-8106-095-9 (print)OAI: oai:DiVA.org:kth-355899DiVA, id: diva2:1911183
Public defence
2024-11-29, T2 (Jacobssonsalen), Hälsovägen 11C, Huddinge, via Zoom: https://kth-se.zoom.us/j/62762467727, Stockholm, 09:00 (English)
Opponent
Supervisors
Note

QC 2024-11-06

Available from: 2024-11-06 Created: 2024-11-06 Last updated: 2024-11-20Bibliographically approved
List of papers
1. Predicting the trabecular bone apparent stiffness tensor with spherical convolutional neural networks
Open this publication in new window or tab >>Predicting the trabecular bone apparent stiffness tensor with spherical convolutional neural networks
2022 (English)In: Bone Reports, E-ISSN 2352-1872, Vol. 16, p. 101179-, article id 101179Article in journal (Refereed) Published
Abstract [en]

The apparent stiffness tensor is relevant for characterizing trabecular bone quality. Previous studies have used morphology-stiffness relationships for estimating the apparent stiffness tensor. In this paper, we propose to train spherical convolutional neural networks (SphCNNs) to estimate this tensor. Information of the edges, trabecular thickness, and spacing are summarized in functions on the unitary sphere used as inputs for the SphCNNs. The concomitant dimensionality reduction makes it possible to train neural networks on relatively small datasets. The predicted tensors were compared to the stiffness tensors computed by using the micro-finite element method (mu FE), which was considered as the gold standard, and models based on fourth-order fabric tensors. Combining edges and trabecular thickness yields significant improvements in the accuracy compared to the methods based on fourth-order fabric tensors. From the results, SphCNNs are promising for replacing the more expensive mu FE stiffness estimations.

Place, publisher, year, edition, pages
Elsevier BV, 2022
Keywords
Apparent stiffness tensor, Trabecular bone, Spherical convolutional neural networks
National Category
Orthopaedics Neurology Medical Laboratory and Measurements Technologies
Identifiers
urn:nbn:se:kth:diva-314204 (URN)10.1016/j.bonr.2022.101179 (DOI)000804615600002 ()35309107 (PubMedID)2-s2.0-85126548178 (Scopus ID)
Note

QC 20220616

Available from: 2022-06-16 Created: 2022-06-16 Last updated: 2024-11-06Bibliographically approved
2. Spherical Convolutional Neural Networks for Survival Rate Prediction in Cancer Patients
Open this publication in new window or tab >>Spherical Convolutional Neural Networks for Survival Rate Prediction in Cancer Patients
2022 (English)In: Frontiers in Oncology, E-ISSN 2234-943X, Vol. 12, article id 870457Article in journal (Refereed) Published
Abstract [en]

ObjectiveSurvival Rate Prediction (SRP) is a valuable tool to assist in the clinical diagnosis and treatment planning of lung cancer patients. In recent years, deep learning (DL) based methods have shown great potential in medical image processing in general and SRP in particular. This study proposes a fully-automated method for SRP from computed tomography (CT) images, which combines an automatic segmentation of the tumor and a DL-based method for extracting rotational-invariant features. MethodsIn the first stage, the tumor is segmented from the CT image of the lungs. Here, we use a deep-learning-based method that entails a variational autoencoder to provide more information to a U-Net segmentation model. Next, the 3D volumetric image of the tumor is projected onto 2D spherical maps. These spherical maps serve as inputs for a spherical convolutional neural network that approximates the log risk for a generalized Cox proportional hazard model. ResultsThe proposed method is compared with 17 baseline methods that combine different feature sets and prediction models using three publicly-available datasets: Lung1 (n=422), Lung3 (n=89), and H&N1 (n=136). We observed comparable C-index scores compared to the best-performing baseline methods in a 5-fold cross-validation on Lung1 (0.59 +/- 0.03 vs. 0.62 +/- 0.04). In comparison, it slightly outperforms all methods in inter-data set evaluation (0.64 vs. 0.63). The best-performing method from the first experiment reduced its performance to 0.61 and 0.62 for Lung3 and H&N1, respectively. DiscussionThe experiments suggest that the performance of spherical features is comparable with previous approaches, but they generalize better when applied to unseen datasets. That might imply that orientation-independent shape features are relevant for SRP. The performance of the proposed method was very similar, using manual and automatic segmentation methods. This makes the proposed model useful in cases where expert annotations are not available or difficult to obtain.

Place, publisher, year, edition, pages
Frontiers Media SA, 2022
Keywords
lung cancer, tumor segmentation, spherical convolutional neural network, survival rate prediction, deep learning, Cox Proportional Hazards, DeepSurv
National Category
Ophthalmology Computer Sciences Obstetrics, Gynecology and Reproductive Medicine
Identifiers
urn:nbn:se:kth:diva-313029 (URN)10.3389/fonc.2022.870457 (DOI)000795556500001 ()35574400 (PubMedID)2-s2.0-85130209481 (Scopus ID)
Note

QC 20220601

Available from: 2022-06-01 Created: 2022-06-01 Last updated: 2024-11-06Bibliographically approved
3. Leveraging Rotational Equivariance for Reinforcement Learning in Tractography
Open this publication in new window or tab >>Leveraging Rotational Equivariance for Reinforcement Learning in Tractography
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Brain tractography involves mapping diffusion-weighted images (DWI) onto streamlines representing neural fibre bundles. Recent research avenues have framed tractography into a reinforcement learning (RL) framework with actor-critic models. However, previous RL-based methods may compromise geometrical relations between the input (DWI) and output (tractogram).

More specifically, 3D rotations applied to the input of RL-based tractography are not adequately reflected in the output, indicating a lack of SO3 equivariance. This study aims to restore the equivariance present in previous non-learning-based methods (e.g., iFOD2 from MRtrix3) to RL-based tractography.

To achieve this, we introduce SO3 equivariant and invariant components for the actors (direction prediction model) and critics (q-value prediction model), respectively. We employ an SE3-equivariant transformer as the next direction prediction function. The fact that both the input DWI and the output directional update can be represented as spherical tensors and transform under representations of SO3 makes this formulation a natural fit for the present problem. 

Another benefit of RL-based tractography is that incorporating local neighbourhoods can help mitigate well-known tractography problems (e.g. kissing, crossing, fanning). Our proposed algorithm extracts neighbourhood information as a predefined graph with spherical signals on the nodes.

The contribution of this work is threefold. First, we discuss rotational equivariance in streamlined tractography on a theoretical level. Second, we propose a method that combines RL-based tractography with an equivariant model. Third, we evaluate the equivariance of the proposed method both locally and globally.

Keywords
Tractography, Reinforcement Learning, Rotational Equivariance, Geometric Deep Learning, SE(3)-Transformer, Diffusion-Weighted MRI
National Category
Medical Image Processing
Research subject
Medical Technology; Computer Science
Identifiers
urn:nbn:se:kth:diva-355875 (URN)
Funder
Knut and Alice Wallenberg Foundation
Note

Acknowledgments: The computations for the experiments were enabled bythe Berzelius resource provided by the Knut and AliceWallenberg Foundation at the National Academic Infras-tructure for Supercomputing in Sweden. This researchhas been partially funded by Digital Futures, the dBrainproject, the Swedish Research Council, Grant No. 2022-03389, and MedTechLabs. The funding sources were notinvolved in the research and preparation of this article.

QC 20241106

Available from: 2024-11-06 Created: 2024-11-06 Last updated: 2024-11-06Bibliographically approved
4. Impact of Tractogram Filtering and Graph Creation for Structural Connectomics in Subjects with Parkinson’s Disease
Open this publication in new window or tab >>Impact of Tractogram Filtering and Graph Creation for Structural Connectomics in Subjects with Parkinson’s Disease
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Structural connectomics derives subject-specific brain connectivity from diffusion-weighted MRI and has potential as a biomarker for clinical Parkinson’s disease (PD) detection. In this study, we applied probabilistic tractography (iFOD2) to derive different types of connectomes and analyzed group discriminability between PD patients and healthy controls from the Parkinson's Progression Markers Initiative (PPMI) dataset (n = 233). Particular emphasis was placed on the streamline filtering stage with SIFT2 and the comparison of different connectivity metrics, including streamline count, fractional anisotropy (FA), axial diffusivity (AD), mean diffusivity (MD), and radial diffusivity (RD). We performed a three-level analysis comprising 1) connection-level statistical analysis, 2) graph theory measures at the node and whole-brain levels, and 3) classification using support vector machines (SVM) and graph neural networks. We did not find any statistical difference at any level after correction for multiple comparisons. Also, the classifiers performed poorly with AUC values close to chance levels. However, we found differences between filtered and unfiltered tractograms at the node level, which could suggest that filtering is a necessary step in structural connectivity analyses. Our findings suggest that structural connectivity analysesfor PD are highly sensitive to specific pipeline configurations and fine-tuning.

Keywords
Tractography, Tractogram filtering, Structural Connectivity, Parkinson’s Disease
National Category
Medical Image Processing
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-355905 (URN)
Note

Funding: This research has been partially funded by Digital Futures, project dBrain, the Swedish Research Council, Grant No.2022-03389, and MedTechLabs. The funding sources had no involvement in the research and preparation of this article.

QC 20241106

Available from: 2024-11-06 Created: 2024-11-06 Last updated: 2024-11-06Bibliographically approved

Open Access in DiVA

summary(18198 kB)108 downloads
File information
File name SUMMARY01.pdfFile size 18198 kBChecksum SHA-512
89de4ae6811cdcba0db62ccd72cfb52d518fcb62d9cdd2e2f770c79fe292b9b5b4253a754be406f85f1692f71b74309f81acbf8a7adf04095d670d1b2dc2f6bc
Type summaryMimetype application/pdf

Authority records

Sinzinger, Fabian

Search in DiVA

By author/editor
Sinzinger, Fabian
By organisation
Medical Imaging
Medical Image Processing

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 542 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf