kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Leveraging Rotational Equivariance for Reinforcement Learning in Tractography
KTH, School of Engineering Sciences in Chemistry, Biotechnology and Health (CBH), Biomedical Engineering and Health Systems, Medical Imaging. Karolinska Institute, Department of Clinical Neurosciences, Neuro Division, Nobels väg 9, D3, 17165, Solna, Sweden. (Biomedical Imaging)ORCID iD: 0000-0003-3502-7441
Sherbrooke Connectivity Imaging Laboratory (SCIL), Department of Computer Sciences, Université de Sherbrooke, 2500, Boul. de l’Université, Sherbrooke, J1K2R1, Québec, Canada.ORCID iD: 0000-0002-3959-8316
KTH, School of Engineering Sciences in Chemistry, Biotechnology and Health (CBH), Biomedical Engineering and Health Systems, Medical Imaging.ORCID iD: 0000-0001-5765-2964
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Brain tractography involves mapping diffusion-weighted images (DWI) onto streamlines representing neural fibre bundles. Recent research avenues have framed tractography into a reinforcement learning (RL) framework with actor-critic models. However, previous RL-based methods may compromise geometrical relations between the input (DWI) and output (tractogram).

More specifically, 3D rotations applied to the input of RL-based tractography are not adequately reflected in the output, indicating a lack of SO3 equivariance. This study aims to restore the equivariance present in previous non-learning-based methods (e.g., iFOD2 from MRtrix3) to RL-based tractography.

To achieve this, we introduce SO3 equivariant and invariant components for the actors (direction prediction model) and critics (q-value prediction model), respectively. We employ an SE3-equivariant transformer as the next direction prediction function. The fact that both the input DWI and the output directional update can be represented as spherical tensors and transform under representations of SO3 makes this formulation a natural fit for the present problem. 

Another benefit of RL-based tractography is that incorporating local neighbourhoods can help mitigate well-known tractography problems (e.g. kissing, crossing, fanning). Our proposed algorithm extracts neighbourhood information as a predefined graph with spherical signals on the nodes.

The contribution of this work is threefold. First, we discuss rotational equivariance in streamlined tractography on a theoretical level. Second, we propose a method that combines RL-based tractography with an equivariant model. Third, we evaluate the equivariance of the proposed method both locally and globally.

Keywords [en]
Tractography, Reinforcement Learning, Rotational Equivariance, Geometric Deep Learning, SE(3)-Transformer, Diffusion-Weighted MRI
National Category
Medical Imaging
Research subject
Medical Technology; Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-355875OAI: oai:DiVA.org:kth-355875DiVA, id: diva2:1910931
Funder
Knut and Alice Wallenberg Foundation
Note

Acknowledgments: The computations for the experiments were enabled bythe Berzelius resource provided by the Knut and AliceWallenberg Foundation at the National Academic Infras-tructure for Supercomputing in Sweden. This researchhas been partially funded by Digital Futures, the dBrainproject, the Swedish Research Council, Grant No. 2022-03389, and MedTechLabs. The funding sources were notinvolved in the research and preparation of this article.

QC 20241106

Available from: 2024-11-06 Created: 2024-11-06 Last updated: 2025-02-09Bibliographically approved
In thesis
1. Geometric Deep Learning for Medical Image Processing Problems
Open this publication in new window or tab >>Geometric Deep Learning for Medical Image Processing Problems
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis consists of four studies, each suggesting incorporating geometric deep-learning methods in medical image-processing pipelines.

The rationale here is that a) conventional Deep learning (DL) models for medical imaging depend highly on the quality and quantity of training data, and b) medical images commonly represent structures with intrinsic geometrical properties that standard DL models do not necessarily respect. 

The first study focuses on predicting stiffness tensors from micro-CT trabecular bone scans. This prediction task requires learning from the complex structures of trabecular bone with limited data available. Our proposed model uses a spherical convolutional neural network (SphCNN) for this purpose. The second study investigates lung cancer survival rate prediction based on image-based features. We evaluate a proposed SphCNN-based pipeline using CT images of non-small cell lung cancer. The third study centres on the stability of the streamline tractography algorithm under arbitrary 3D rotations. We propose integrating an SE(3)-equivariant transformer model into the tractography framework to preserve rotational equivariance. The fourth study evaluates a structural connectivity pipeline in combination with tractography filtering, subsequent classification and analysis. The pipeline is evaluated based on its ability to identify group differences in brain connectivity related to Parkinson's disease. 

Abstract [sv]

I denna avhandling presenteras fyra studier som tillsammans utforskar en integration av geometriska djupinlärningsmetoder i bildbehandling för medicinska tillämpningar.

Motivationen till detta är att a) konventionella djupinlärningsmodeller för medicinsk avbildning i hög grad är beroende av träningsdatans mängd och kvalitet, samt b) medicinska bilder ofta avbildar strukturer med inneboende geometriska egenskaper som traditionella djupinlärningsmodeller inte nödvändigtvis tar hänsyn till.

Den första studien fokuserar på att förutsäga styvhetstensorer baserat på mikro-CT-bilder av trabekulärt ben. Denna förutsägelse kräver inlärning från de komplexa strukturerna i trabekulärt ben med begränsad tillgång på data. Vår föreslagna modell använder ett sfäriskt faltningseuronnät  (SphCNN) för detta ändamål. Den andra studien undersöker överlevnadsprognoser vid lungcancer utifrån bildbaserade egenskaper. Vi utvärderar en föreslagen pipeline, baserad på SphCNN, med CT-bilder av icke-småcellig lungcancer. Den tredje studien fokuserar på stabiliteten hos en traktografialgoritm för riktade fibrer under godtyckliga tredimensionella rotationer. Vi föreslår att integrera en SE(3)-ekvivariant transformermodell i traktografins ramverk för att bevara rotations-ekvivariansen. Den fjärde studien utvärderar en pipeline för strukturell konnektivitet i kombination med traktogramfiltrering, klassificering och analys. Pipelinen utvärderas utifrån dess förmåga att identifiera skillnader i hjärnans konnektivitet kopplat till Parkinsons sjukdom.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. p. 114
Series
TRITA-CBH-FOU ; 2024:48
Keywords
Geometric Deep Learning, Medical Image Processing, Diffusion Brain MRI, Spherical Convolutional Neural Networks, Structural Connectivity, Bone Stiffness Tensor Prediction, Lung Cancer Survival Rate Prediction, Tractography, Geometriskt djupinlärning, Medicinsk bildbehandling, Diffusions hjärn-MRI, Sfäriska konvolutionella neurala nätverk, Strukturell konnektivitet, Styvhetstensorprediktion för ben, Lungcancersöverlevnadsprognos, Traktografi
National Category
Medical Imaging
Research subject
Medical Technology
Identifiers
urn:nbn:se:kth:diva-355899 (URN)978-91-8106-095-9 (ISBN)
Public defence
2024-11-29, T2 (Jacobssonsalen), Hälsovägen 11C, Huddinge, via Zoom: https://kth-se.zoom.us/j/62762467727, Stockholm, 09:00 (English)
Opponent
Supervisors
Note

QC 2024-11-06

Available from: 2024-11-06 Created: 2024-11-06 Last updated: 2026-01-13Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Sinzinger, FabianMoreno, Rodrigo

Search in DiVA

By author/editor
Sinzinger, FabianThéberge, AntoineMoreno, Rodrigo
By organisation
Medical Imaging
Medical Imaging

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 214 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf