kth.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 58) Show all publications
Banert, S., Rudzusika, J., Öktem, O. & Adler, J. (2024). Accelerated Forward-Backward Optimization Using Deep Learning. SIAM Journal on Optimization, 34(2), 1236-1263
Open this publication in new window or tab >>Accelerated Forward-Backward Optimization Using Deep Learning
2024 (English)In: SIAM Journal on Optimization, ISSN 1052-6234, E-ISSN 1095-7189, Vol. 34, no 2, p. 1236-1263Article in journal (Refereed) Published
Abstract [en]

We propose several deep -learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward -backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step -size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.

Place, publisher, year, edition, pages
Society for Industrial & Applied Mathematics (SIAM), 2024
Keywords
convex optimization, deep learning, proximal-gradient algorithm, inverse problems
National Category
Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-345940 (URN)10.1137/22M1532548 (DOI)001196808600001 ()2-s2.0-85190534496 (Scopus ID)
Note

QC 20240426

Available from: 2024-04-26 Created: 2024-04-26 Last updated: 2024-04-26Bibliographically approved
Mukherjee, S., Dittmer, S., Shumaylov, Z., Lunz, S., Öktem, O. & Schönlieb, C. B. (2024). DATA-DRIVEN CONVEX REGULARIZERS FOR INVERSE PROBLEMS. In: 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024 - Proceedings: . Paper presented at 49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024, Seoul, Korea, Apr 14 2024 - Apr 19 2024 (pp. 13386-13390). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>DATA-DRIVEN CONVEX REGULARIZERS FOR INVERSE PROBLEMS
Show others...
2024 (English)In: 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024 - Proceedings, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 13386-13390Conference paper, Published paper (Refereed)
Abstract [en]

We propose to learn a data-adaptive convex regularizer, which is parameterized using an input-convex neural network (ICNN), for variational image reconstruction. The regularizer parameters are learned adversarially by telling apart clean images from the artifact-ridden ones in a training dataset. Convexity of the regularizer is theoretically and practically important since (i) one can establish well-posedness guarantees for the corresponding variational reconstruction problem and (ii) devise provably convergent optimization algorithms for reconstruction. In particular, the resulting method is shown to be convergent in the sense of regularization and can be solved provably using a gradient-based solver. To demonstrate the performance of our approach for solving inverse problems, we consider deblurring natural images and reconstruction in X-ray computed tomography (CT) and show that the proposed convex regularizer is on par with and sometimes superior to state-of-the-art classical and data-driven techniques for inverse problems, especially with severely ill-posed forward operators (such as in limited-angle tomography).

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Series
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, ISSN 1520-6149
Keywords
data-driven regularization, input-convex neural networks, Inverse problems, variational imaging
National Category
Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-348292 (URN)10.1109/ICASSP48485.2024.10447719 (DOI)2-s2.0-85195377115 (Scopus ID)
Conference
49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024, Seoul, Korea, Apr 14 2024 - Apr 19 2024
Note

QC 20240626

Part of ISBN 979-835034485-1

Available from: 2024-06-20 Created: 2024-06-20 Last updated: 2024-06-26Bibliographically approved
Buddenkotte, T., Escudero Sanchez, L., Crispin-Ortuzar, M., Woitek, R., McCague, C., Brenton, J. D., . . . Rundo, L. (2023). Calibrating ensembles for scalable uncertainty quantification in deep learning-based medical image segmentation. Computers in Biology and Medicine, 163, Article ID 107096.
Open this publication in new window or tab >>Calibrating ensembles for scalable uncertainty quantification in deep learning-based medical image segmentation
Show others...
2023 (English)In: Computers in Biology and Medicine, ISSN 0010-4825, E-ISSN 1879-0534, Vol. 163, article id 107096Article in journal (Refereed) Published
Abstract [en]

Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, quantifying the uncertainty of the models can play a critical role for example in active learning or machine human interaction. Uncertainty quantification is especially difficult when using deep learning-based models, which are the state-of-the-art in many imaging applications. The current uncertainty quantification approaches do not scale well in high-dimensional real-world problems. Scalable solutions often rely on classical techniques, such as dropout, during inference or training ensembles of identical models with different random seeds to obtain a posterior distribution. In this paper, we present the following contributions. First, we show that the classical approaches fail to approximate the classification probability. Second, we propose a scalable and intuitive framework for uncertainty quantification in medical image segmentation that yields measurements that approximate the classification probability. Third, we suggest the usage of k-fold cross-validation to overcome the need for held out calibration data. Lastly, we motivate the adoption of our method in active learning, creating pseudo-labels to learn from unlabeled images and human–machine collaboration.

Place, publisher, year, edition, pages
Elsevier Ltd, 2023
Keywords
Deep learning, Segmentation, Uncertainty quantification
National Category
Computer Vision and Robotics (Autonomous Systems) Radiology, Nuclear Medicine and Medical Imaging
Identifiers
urn:nbn:se:kth:diva-331437 (URN)10.1016/j.compbiomed.2023.107096 (DOI)001028875900001 ()37302375 (PubMedID)2-s2.0-85161555981 (Scopus ID)
Note

QC 20230710

Available from: 2023-07-10 Created: 2023-07-10 Last updated: 2023-08-10Bibliographically approved
Buddenkotte, T., Rundo, L., Woitek, R., Sanchez, L. E., Beer, L., Crispin-Ortuzar, M., . . . Schonlieb, C.-B. (2023). Deep learning-based segmentation of multisite disease in ovarian cancer. EUROPEAN RADIOLOGY EXPERIMENTAL, 7(1), Article ID 77.
Open this publication in new window or tab >>Deep learning-based segmentation of multisite disease in ovarian cancer
Show others...
2023 (English)In: EUROPEAN RADIOLOGY EXPERIMENTAL, ISSN 2509-9280, Vol. 7, no 1, article id 77Article in journal (Refereed) Published
Abstract [en]

Purpose: To determine if pelvic/ovarian and omental lesions of ovarian cancer can be reliably segmented on computed tomography (CT) using fully automated deep learning-based methods.

Methods: A deep learning model for the two most common disease sites of high-grade serous ovarian cancer lesions (pelvis/ovaries and omentum) was developed and compared against the well-established “no-new-Net” framework and unrevised trainee radiologist segmentations. A total of 451 CT scans collected from four different institutions were used for training (n = 276), evaluation (n = 104) and testing (n = 71) of the methods. The performance was evaluated using the Dice similarity coefficient (DSC) and compared using a Wilcoxon test.

Results: Our model outperformed no-new-Net for the pelvic/ovarian lesions in cross-validation, on the evaluation and test set by a significant margin (p values being 4 × 10–7, 3 × 10–4, 4 × 10–2, respectively), and for the omental lesions on the evaluation set (p = 1 × 10–3). Our model did not perform significantly differently in segmenting pelvic/ovarian lesions (p = 0.371) compared to a trainee radiologist. On an independent test set, the model achieved a DSC performance of 71 ± 20 (mean ± standard deviation) for pelvic/ovarian and 61 ± 24 for omental lesions.

Conclusion: Automated ovarian cancer segmentation on CT scans using deep neural networks is feasible and achieves performance close to a trainee-level radiologist for pelvic/ovarian lesions.

Relevance statement: Automated segmentation of ovarian cancer may be used by clinicians for CT-based volumetric assessments and researchers for building complex analysis pipelines.

Key points:

  • The first automated approach for pelvic/ovarian and omental ovarian cancer lesion segmentation on CT images has been presented.
  • Automated segmentation of ovarian cancer lesions can be comparable with manual segmentation of trainee radiologists.
  • Careful hyperparameter tuning can provide models significantly outperforming strong state-of-the-art baselines. Graphical Abstract: [Figure not available: see fulltext.]
Place, publisher, year, edition, pages
Springer Nature, 2023
Keywords
Deep learning, Omentum, Ovarian Neoplasms, Tomography (x-ray computed), Pelvis
National Category
Cancer and Oncology Medical Image Processing
Identifiers
urn:nbn:se:kth:diva-341569 (URN)10.1186/s41747-023-00388-z (DOI)001116858300001 ()38057616 (PubMedID)2-s2.0-85178885749 (Scopus ID)
Note

QC 20231222

Available from: 2023-12-22 Created: 2023-12-22 Last updated: 2023-12-22Bibliographically approved
Sanchez, L. E., Buddenkotte, T., Al Sa'd, M., McCague, C., Darcy, J., Rundo, L., . . . Öktem, O. (2023). Integrating Artificial Intelligence Tools in the Clinical Research Setting: The Ovarian Cancer Use Case. Diagnostics, 13(17), Article ID 2813.
Open this publication in new window or tab >>Integrating Artificial Intelligence Tools in the Clinical Research Setting: The Ovarian Cancer Use Case
Show others...
2023 (English)In: Diagnostics, ISSN 2075-4418, Vol. 13, no 17, article id 2813Article in journal (Refereed) Published
Abstract [en]

Artificial intelligence (AI) methods applied to healthcare problems have shown enormous potential to alleviate the burden of health services worldwide and to improve the accuracy and reproducibility of predictions. In particular, developments in computer vision are creating a paradigm shift in the analysis of radiological images, where AI tools are already capable of automatically detecting and precisely delineating tumours. However, such tools are generally developed in technical departments that continue to be siloed from where the real benefit would be achieved with their usage. Significant effort still needs to be made to make these advancements available, first in academic clinical research and ultimately in the clinical setting. In this paper, we demonstrate a prototype pipeline based entirely on open-source software and free of cost to bridge this gap, simplifying the integration of tools and models developed within the AI community into the clinical research setting, ensuring an accessible platform with visualisation applications that allow end-users such as radiologists to view and interact with the outcome of these AI tools.

Place, publisher, year, edition, pages
MDPI, 2023
Keywords
artificial intelligence, cancer research, imaging, clinical integration, radiomics
National Category
Computer Sciences Radiology, Nuclear Medicine and Medical Imaging
Identifiers
urn:nbn:se:kth:diva-337027 (URN)10.3390/diagnostics13172813 (DOI)001061016200001 ()37685352 (PubMedID)2-s2.0-85170386154 (Scopus ID)
Note

QC 20230922

Available from: 2023-09-22 Created: 2023-09-22 Last updated: 2023-09-22Bibliographically approved
Mukherjee, S., Hauptmann, A., Öktem, O., Pereyra, M. & Schonlieb, C.-B. (2023). Learned Reconstruction Methods With Convergence Guarantees: A survey of concepts and applications. IEEE signal processing magazine (Print), 40(1), 164-182
Open this publication in new window or tab >>Learned Reconstruction Methods With Convergence Guarantees: A survey of concepts and applications
Show others...
2023 (English)In: IEEE signal processing magazine (Print), ISSN 1053-5888, E-ISSN 1558-0792, Vol. 40, no 1, p. 164-182Article in journal, Editorial material (Refereed) Published
Abstract [en]

In recent years, deep learning has achieved remarkable empirical success for image reconstruction. This has catalyzed an ongoing quest for the precise characterization of the correctness and reliability of data-driven methods in critical use cases, for instance, in medical imaging. Notwithstanding the excellent performance and efficacy of deep learning-based methods, concerns have been raised regarding the approaches' stability, or lack thereof, with serious practical implications. Significant advances have been made in recent years to unravel the inner workings of data-driven image recovery methods, challenging their widely perceived black-box nature. In this article, we specify relevant notions of convergence for data-driven image reconstruction, which forms the basis of a survey of learned methods with mathematically rigorous reconstruction guarantees. An example that is highlighted is the role of input-convex neural networks (ICNNs), offering the possibility to combine the power of deep learning with classical convex regularization theory for devising methods that are provably convergent. This survey article is aimed at both methodological researchers seeking to advance the frontiers of our understanding of data-driven image reconstruction methods as well as practitioners by providing an accessible description of useful convergence concepts and by placing some of the existing empirical practices on a solid mathematical foundation.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
Deep learning, Learning systems, Neural networks, Closed box, Reconstruction algorithms, Image reconstruction, Reliability
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-326657 (URN)10.1109/MSP.2022.3207451 (DOI)000966460000001 ()
Note

QC 20240312

Available from: 2023-05-08 Created: 2023-05-08 Last updated: 2024-06-17Bibliographically approved
Diepeveen, W., Lellmann, J., Öktem, O. & Schonlieb, C. B. (2023). Regularizing Orientation Estimation in Cryogenic Electron Microscopy Three-Dimensional Map Refinement through Measure-Based Lifting over Riemannian Manifolds. SIAM Journal on Imaging Sciences, 16(3), 1440-1490
Open this publication in new window or tab >>Regularizing Orientation Estimation in Cryogenic Electron Microscopy Three-Dimensional Map Refinement through Measure-Based Lifting over Riemannian Manifolds
2023 (English)In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 16, no 3, p. 1440-1490Article in journal (Refereed) Published
Abstract [en]

Motivated by the trade-off between noise robustness and data consistency for joint three-imensional (3D) map reconstruction and rotation estimation in single particle cryogenic-electron microscopy (Cryo-EM), we propose ellipsoidal support lifting (ESL), a measure-based lifting scheme for regularizing and approximating the global minimizer of a smooth function over a Riemannian manifold. Under a uniqueness assumption on the minimizer we show several theoretical results, in particular well-posedness of the method and an error bound due to the induced bias with respect to the global minimizer. Additionally, we use the developed theory to integrate the measure-based lifting scheme into an alternating update method for joint homogeneous 3D map reconstruction and rotation estimation, where typically tens of thousands of manifold-valued minimization problems have to be solved and where regularization is necessary because of the high noise levels in the data. The joint recovery method is used to test both the theoretical predictions and algorithmic performance through numerical experiments with Cryo-EM data. In particular, the induced bias due to the regularizing effect of ESL empirically estimates better rotations, i.e., rotations closer to the ground truth, than global optimization would.

Place, publisher, year, edition, pages
Society for Industrial & Applied Mathematics (SIAM), 2023
Keywords
cryo-electron microscopy, global optimization, nonconvex optimization, regularization, Riemannian optimization, rotation estimation
National Category
Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-341940 (URN)10.1137/22M1520773 (DOI)001165605800004 ()2-s2.0-85180366036 (Scopus ID)
Note

QC 20240301

Available from: 2024-01-08 Created: 2024-01-08 Last updated: 2024-04-25Bibliographically approved
Esteve-Yague, C., Diepeveen, W., Öktem, O. & Schonlieb, C.-B. (2023). Spectral decomposition of atomic structures in heterogeneous cryo-EM. Inverse Problems, 39(3), 034003, Article ID 034003.
Open this publication in new window or tab >>Spectral decomposition of atomic structures in heterogeneous cryo-EM
2023 (English)In: Inverse Problems, ISSN 0266-5611, E-ISSN 1361-6420, Vol. 39, no 3, p. 034003-, article id 034003Article in journal (Refereed) Published
Abstract [en]

We consider the problem of recovering the three-dimensional atomic structure of a flexible macromolecule from a heterogeneous cryogenic electron microscopy (cryo-EM) dataset. The dataset contains noisy tomographic projections of the electrostatic potential of the macromolecule, taken from different viewing directions, and in the heterogeneous case, each cryo-EM image corresponds to a different conformation of the macromolecule. Under the assumption that the macromolecule can be modelled as a chain, or discrete curve (as it is for instance the case for a protein backbone with a single chain of amino-acids), we introduce a method to estimate the deformation of the atomic model with respect to a given conformation, which is assumed to be known a priori. Our method consists on estimating the torsion and bond angles of the atomic model in each conformation as a linear combination of the eigenfunctions of the Laplace operator in the manifold of conformations. These eigenfunctions can be approximated by means of a well-known technique in manifold learning, based on the construction of a graph Laplacian using the cryo-EM dataset. Finally, we test our approach with synthetic datasets, for which we recover the atomic model of two-dimensional and three-dimensional flexible structures from simulated cryo-EM images.

Place, publisher, year, edition, pages
IOP Publishing, 2023
Keywords
heterogeneous cryo-EM, atomic structure decomposition, graph Laplacian, tomographic reconstruction
National Category
Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-324760 (URN)10.1088/1361-6420/acb2ba (DOI)000921748100001 ()2-s2.0-85147142754 (Scopus ID)
Note

QC 20230315

Available from: 2023-03-15 Created: 2023-03-15 Last updated: 2023-03-15Bibliographically approved
Eguizabal, A., Öktem, O. & Persson, M. (2022). A deep learning one-step solution to material image reconstruction in photon counting spectral CT. In: Proceedings Volume 12031, Medical Imaging 2022: Physics of Medical Imaging: . Paper presented at SPIE Medical Imaging 2022: Physics of Medical Imaging. SPIE-Intl Soc Optical Eng
Open this publication in new window or tab >>A deep learning one-step solution to material image reconstruction in photon counting spectral CT
2022 (English)In: Proceedings Volume 12031, Medical Imaging 2022: Physics of Medical Imaging, SPIE-Intl Soc Optical Eng , 2022Conference paper, Published paper (Refereed)
Place, publisher, year, edition, pages
SPIE-Intl Soc Optical Eng, 2022
National Category
Medical Image Processing
Identifiers
urn:nbn:se:kth:diva-312893 (URN)10.1117/12.2612426 (DOI)000836294000033 ()2-s2.0-85131211131 (Scopus ID)
Conference
SPIE Medical Imaging 2022: Physics of Medical Imaging
Note

QC 20220921

Available from: 2022-05-24 Created: 2022-05-24 Last updated: 2022-09-21Bibliographically approved
Rudzusika, J., Koehler, T. & Öktem, O. (2022). Deep Learning-Based Dictionary Learning and Tomographic Image Reconstruction. SIAM Journal on Imaging Sciences, 15(4), 1729-1764
Open this publication in new window or tab >>Deep Learning-Based Dictionary Learning and Tomographic Image Reconstruction
2022 (English)In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 15, no 4, p. 1729-1764Article in journal (Refereed) Published
Abstract [en]

This work presents an approach for image reconstruction in clinical low-dose tomography that combines principles from sparse signal processing with ideas from deep learning. First, we describe sparse signal representation in terms of dictionaries from a statistical perspective and interpret dictionary learning as a process of aligning the distribution that arises from a generative model with the empirical distribution of true signals. As a result, we can see that sparse coding with learned dictionaries resembles a specific variational autoencoder, where the encoder is a sparse coding algorithm and the decoder is a linear function. Next, we show that dictionary learning can also benefit from computational advancements introduced in the context of deep learning, such as parallelism and stochastic optimization. Finally, we show that regularization by dictionaries achieves competitive performance in computed tomography reconstruction compared to state-of-the-art model-based and data-driven approaches, while being unsupervised with respect to tomographic data.

Place, publisher, year, edition, pages
Society for Industrial & Applied Mathematics (SIAM), 2022
Keywords
dictionary learning, generative model, deep learning, image reconstruction, computed tomography
National Category
Signal Processing
Identifiers
urn:nbn:se:kth:diva-323412 (URN)10.1137/21M1445697 (DOI)000903981200005 ()
Note

QC 20230201

Available from: 2023-02-01 Created: 2023-02-01 Last updated: 2024-04-25Bibliographically approved
Projects
Mathematical methods for 3D electron microscopy [2020-03107_VR]; Uppsala University
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-1118-6483

Search in DiVA

Show all publications