kth.sePublications
Change search
Refine search result
12 1 - 50 of 59
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Adler, Jonas
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). DeepMind, 6 Pancras Square, London, N1C 4AG, United Kingdom.
    Lunz, Sebastian
    Univ Cambridge, Ctr Math Sci, Cambridge CB3 0WA, England..
    Verdier, Olivier
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Department of Computing, Mathematics and Physics, Western Norway University of Applied Sciences, Bergen, Norway.
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Ctr Math Sci, Cambridge CB3 0WA, England..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Division of Scientific Computing, Department of Information Technology, Uppsala University.
    Task adapted reconstruction for inverse problems2022In: Inverse Problems, ISSN 0266-5611, E-ISSN 1361-6420, Vol. 38, no 7, article id 075006Article in journal (Refereed)
    Abstract [en]

    The paper considers the problem of performing a post-processing task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and post-processing as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the post-processing task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any post-processing that can be encoded as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation.

  • 2.
    Adler, Jonas
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Elekta.
    Lunz, Sebastian
    Centre for Mathematical Sciences, University of Cambridge, Cambridge CB3 0WA, United Kingdom.
    Verdier, Olivier
    Department of Mathematics, KTH Royal Institute of Technology, 100 44 Stockholm, Sweden ; Department of Computing, Mathematics and Physics, Western Norway University of Applied Sciences, Bergen, Norway.
    Schönlieb, Carola-Bibiane
    Centre for Mathematical Sciences, University of Cambridge, Cambridge CB3 0WA, United Kingdom.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Task adapted reconstruction for inverse problemsManuscript (preprint) (Other academic)
    Abstract [en]

    The paper considers the problem of performing a task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and task as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any task that is encodable as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation.

    Download full text (pdf)
    fulltext
  • 3.
    Adler, Jonas
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Elekta, Box 7593, 103 93 Stockholm, Sweden.
    Ringh, Axel
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Optimization and Systems Theory.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Karlsson, Johan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Optimization and Systems Theory.
    Learning to solve inverse problems using Wasserstein lossManuscript (preprint) (Other academic)
    Abstract [en]

    We propose using the Wasserstein loss for training in inverse problems. In particular, we consider a learned primal-dual reconstruction scheme for ill-posed inverse problems using the Wasserstein distance as loss function in the learning. This is motivated by miss-alignments in training data, which when using standard mean squared error loss could severely degrade reconstruction quality. We prove that training with the Wasserstein loss gives a reconstruction operator that correctly compensates for miss-alignments in certain cases, whereas training with the mean squared error gives a smeared reconstruction. Moreover, we demonstrate these effects by training a reconstruction algorithm using both mean squared error and optimal transport loss for a problem in computerized tomography.

  • 4.
    Adler, Jonas
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Elekta.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Deep Bayesian InversionManuscript (preprint) (Other academic)
    Abstract [en]

    Characterizing statistical properties of solutions of inverse problems is essential for decision making. Bayesian inversion offers a tractable framework for this purpose, but current approaches are computationally unfeasible for most realistic imaging applications in the clinic. We introduce two novel deep learning based methods for solving large-scale inverse problems using Bayesian inversion: a sampling based method using a WGAN with a novel mini-discriminator and a direct approach that trains a neural network using a novel loss function. The performance of both methods is demonstrated on image reconstruction in ultra low dose 3D helical CT. We compute the posterior mean and standard deviation of the 3D images followed by a hypothesis test to assess whether a "dark spot" in the liver of a cancer stricken patient is present. Both methods are computationally efficient and our evaluation shows very promising performance that clearly supports the claim that Bayesian inversion is usable for 3D imaging in time critical applications.

    Download full text (pdf)
    fulltext
  • 5.
    Adler, Jonas
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Elekta Instrument AB, Stockholm, Sweden.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Learned Primal-Dual Reconstruction2018In: IEEE Transactions on Medical Imaging, ISSN 0278-0062, E-ISSN 1558-254X, Vol. 37, no 6, p. 1322-1332Article in journal (Refereed)
    Abstract [en]

    We propose the Learned Primal-Dual algorithm for tomographic reconstruction. The algorithm accounts for a (possibly non-linear) forward operator in a deep neural network by unrolling a proximal primal-dual optimization method, but where the proximal operators have been replaced with convolutional neural networks. The algorithm is trained end-to-end, working directly from raw measured data and it does not depend on any initial reconstruction such as filtered back-projection (FBP). We compare performance of the proposed method on low dose computed tomography reconstruction against FBP, total variation (TV), and deep learning based post-processing of FBP. For the Shepp-Logan phantom we obtain >6 dB peak signal to noise ratio improvement against all compared methods. For human phantoms the corresponding improvement is 6.6 dB over TV and 2.2 dB over learned post-processing along with a substantial improvement in the structural similarity index. Finally, our algorithm involves only ten forward-back-projection computations, making the method feasible for time critical clinical applications.

  • 6.
    Adler, Jonas
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Solving ill-posed inverse problems using iterative deep neural networks2017In: Inverse Problems, ISSN 0266-5611, E-ISSN 1361-6420, Vol. 33, no 12, article id 124007Article in journal (Refereed)
    Abstract [en]

    We propose a partially learned approach for the solution of ill-posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularisation theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularising functional. The method results in a gradient-like iterative scheme, where the 'gradient' component is learned using a convolutional network that includes the gradients of the data discrepancy and regulariser as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against filtered backprojection and total variation reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the total variation reconstruction while being significantly faster, giving reconstructions of 512 x 512 pixel images in about 0.4 s using a single graphics processing unit (GPU).

  • 7. Andrade-Loarca, H.
    et al.
    Kutyniok, G.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Petersen, P.
    Extraction of digital wavefront sets using applied harmonic analysis and deep neural networks2019In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 12, no 4, p. 1936-1966Article in journal (Refereed)
    Abstract [en]

    Microlocal analysis provides deep insight into singularity structures and is often crucial for solving inverse problems, predominately, in imaging sciences. Of particular importance is the analysis of wavefront sets and the correct extraction of those. In this paper, we introduce the first algorithmic approach to extract the wavefront set of images, which combines data-based and model-based methods. Based on a celebrated property of the shearlet transform to unravel information on the wavefront set, we extract the wavefront set of an image by first applying a discrete shearlet transform and then feeding local patches of this transform to a deep convolutional neural network trained on labeled data. The resulting algorithm outperforms all competing algorithms in edge-orientation and ramp-orientation detection.

  • 8.
    Andrade-Loarca, Hector
    et al.
    Tech Univ Berlin, Inst Math, D-10623 Berlin, Germany..
    Kutyniok, Gitta
    Tech Univ Berlin, Inst Math, D-10623 Berlin, Germany.;Tech Univ Berlin, Fak Elektrotech & Informat, D-10587 Berlin, Germany.;Univ Tromso, Dept Phys & Technol, Tromso, Norway..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Shearlets as feature extractor for semantic edge detection: the model-based and data-driven realm2020In: Proceedings of the Royal Society. Mathematical, Physical and Engineering Sciences, ISSN 1364-5021, E-ISSN 1471-2946, Vol. 476, no 2243, article id 20190841Article in journal (Refereed)
    Abstract [en]

    Semantic edge detection has recently gained a lot of attention as an image-processing task, mainly because of its wide range of real-world applications. This is based on the fact that edges in images contain most of the semantic information. Semantic edge detection involves two tasks, namely pure edge detection and edge classification. Those are in fact fundamentally distinct in terms of the level of abstraction that each task requires. This fact is known as the distracted supervision paradox and limits the possible performance of a supervised model in semantic edge detection. In this work, we will present a novel hybrid method that is based on a combination of the model-based concept of shearlets, which provides probably optimally sparse approximations of a model class of images, and the data-driven method of a suitably designed convolutional neural network. We show that it avoids the distracted supervision paradox and achieves high performance in semantic edge detection. In addition, our approach requires significantly fewer parameters than a pure data-driven approach. Finally, we present several applications such as tomographic reconstruction and show that our approach significantly outperforms former methods, thereby also indicating the value of such hybrid methods for biomedical imaging.

  • 9.
    Andrade-Loarca, Hector
    et al.
    Ludwig Maximilians Univ Munchen, Dept Math, D-80333 Munich, Germany..
    Kutyniok, Gitta
    Ludwig Maximilians Univ Munchen, Dept Math, D-80333 Munich, Germany.;Univ Tromso, Dept Phys & Technol, N-9019 Tromso, Norway..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Department of Information Technology, Division of Scientific Computing, Uppsala University, Uppsala, Sweden.
    Petersen, Philipp
    Univ Vienna, Fac Math, A-1090 Vienna, Austria.;Univ Vienna, Res Network Data Sci, A-1090 Vienna, Austria..
    Deep microlocal reconstruction for limited-angle tomography2022In: Applied and Computational Harmonic Analysis, ISSN 1063-5203, E-ISSN 1096-603X, Vol. 59, p. 155-197Article in journal (Refereed)
    Abstract [en]

    We present a deep-learning-based algorithm to jointly solve a reconstruction problem and a wavefront set extraction problem in tomographic imaging. The algorithm is based on a recently developed digital wavefront set extractor as well as the well-known microlocal canonical relation for the Radon transform. We use the wavefront set information about x-ray data to improve the reconstruction by requiring that the underlying neural networks simultaneously extract the correct ground truth wavefront set and ground truth image. As a necessary theoretical step, we identify the digital microlocal canonical relations for deep convolutional residual neural networks. We find strong numerical evidence for the effectiveness of this approach.

  • 10.
    Arridge, Simon
    et al.
    UCL, Dept Comp Sci, Gower St, London WC1E 6BT, England..
    Maass, Peter
    Univ Bremen, Dept Math, Postfach 330 440, D-28344 Bremen, Germany..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Dept Appl Math & Theoret Phys, Wilberforce Rd, Cambridge CB3 0WA, England..
    Solving inverse problems using data-driven models2019In: Acta Numerica, ISSN 0962-4929, E-ISSN 1474-0508, Vol. 28, p. 1-174Article in journal (Refereed)
    Abstract [en]

    Recent research in inverse problems seeks to develop a mathematically coherent foundation for combining data-driven models, and in particular those based on deep learning, with domain-specific knowledge contained in physical-analytical models. The focus is on solving ill-posed inverse problems that are at the core of many challenging applications in the natural sciences, medicine and life sciences, as well as in engineering and industrial applications. This survey paper aims to give an account of some of the main contributions in data-driven inverse problems.

  • 11.
    Aspri, A.
    et al.
    Johann Radon Institute Linz Austria.
    Banert, Sebastian
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM. KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Scherzer, O.
    Johann Radon Institute Linz Austria.
    A Data-Driven Iteratively Regularized Landweber Iteration2020In: Numerical Functional Analysis and Optimization, ISSN 0163-0563, E-ISSN 1532-2467Article in journal (Refereed)
    Abstract [en]

    We derive and analyze a new variant of the iteratively regularized Landweber iteration, for solving linear and nonlinear ill-posed inverse problems. The method takes into account training data, which are used to estimate the interior of a black box, which is used to define the iteration process. We prove convergence and stability for the scheme in infinite dimensional Hilbert spaces. These theoretical results are complemented by some numerical experiments for solving linear inverse problems for the Radon transform and a nonlinear inverse problem for Schlieren tomography. 

  • 12.
    Banert, Sebastian
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Ringh, Axel
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Optimization and Systems Theory.
    Adler, Jonas
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Elekta, Box 7593, S-10393 Stockholm, Sweden..
    Karlsson, Johan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Optimization and Systems Theory.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Data-driven nonsmooth optimization2020In: SIAM Journal on Optimization, ISSN 1052-6234, E-ISSN 1095-7189, Vol. 30, no 1, p. 102-131Article in journal (Refereed)
    Abstract [en]

    In this work, we consider methods for solving large-scale optimization problems with a possibly nonsmooth objective function. The key idea is to first parametrize a class of optimization methods using a generic iterative scheme involving only linear operations and applications of proximal operators. This scheme contains some modern primal-dual first-order algorithms like the Douglas-Rachford and hybrid gradient methods as special cases. Moreover, we show weak convergence of the iterates to an optimal point for a new method which also belongs to this class. Next, we interpret the generic scheme as a neural network and use unsupervised training to learn the best set of parameters for a specific class of objective functions while imposing a fixed number of iterations. In contrast to other approaches of "learning to optimize," we present an approach which learns parameters only in the set of convergent schemes. Finally, we illustrate the approach on optimization problems arising in tomographic reconstruction and image deconvolution, and train optimization algorithms for optimal performance given a fixed number of iterations.

  • 13.
    Banert, Sebastian
    et al.
    Lund Univ, Dept Automat Control, S-22363 Lund, Sweden..
    Rudzusika, Jevgenija
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Adler, Jonas
    DeepMind, London, England..
    Accelerated Forward-Backward Optimization Using Deep Learning2024In: SIAM Journal on Optimization, ISSN 1052-6234, E-ISSN 1095-7189, Vol. 34, no 2, p. 1236-1263Article in journal (Refereed)
    Abstract [en]

    We propose several deep -learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward -backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step -size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.

  • 14.
    Bergstrand, Jan
    et al.
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics.
    Xu, Lei
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics. Royal Inst Technol KTH, Dept Appl Phys, Albanova Univ Ctr, Expt Biomol Phys, SE-10691 Stockholm, Sweden..
    Miao, Xinyan
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics. Royal Inst Technol KTH, Dept Appl Phys, Albanova Univ Ctr, Expt Biomol Phys, SE-10691 Stockholm, Sweden..
    Li, Nailin
    Karolinska Inst, Dept Med Solna, Karolinska Univ Hosp Solna, Clin Pharmacol, L7 03, SE-17176 Stockholm, Sweden..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Franzen, Bo
    Karolinska Inst, Dept Oncol Pathol, Karolinska Univ Hosp, K7,Z1 00, S-17176 Stockholm, Sweden..
    Auer, Gert
    Karolinska Inst, Dept Oncol Pathol, Karolinska Univ Hosp, K7,Z1 00, S-17176 Stockholm, Sweden..
    Lomnytska, Marta
    Karolinska Inst, Dept Oncol Pathol, Karolinska Univ Hosp, K7,Z1 00, S-17176 Stockholm, Sweden.;Acad Univ Hosp, Dept Obstet & Gynaecol, SE-75185 Uppsala, Sweden.;Uppsala Univ, Inst Women & Child Hlth, SE-75185 Uppsala, Sweden..
    Widengren, Jerker
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics.
    Super-resolution microscopy can identify specific protein distribution patterns in platelets incubated with cancer cells2019In: Nanoscale, ISSN 2040-3364, E-ISSN 2040-3372, Vol. 11, no 20, p. 10023-10033Article in journal (Refereed)
    Abstract [en]

    Protein contents in platelets are frequently changed upon tumor development and metastasis. However, how cancer cells can influence protein-selective redistribution and release within platelets, thereby promoting tumor development, remains largely elusive. With fluorescence-based super-resolution stimulated emission depletion (STED) imaging we reveal how specific proteins, implicated in tumor progression and metastasis, re-distribute within platelets, when subject to soluble activators (thrombin, adenosine diphosphate and thromboxane A2), and when incubated with cancer (MCF-7, MDA-MB-231, EFO21) or non-cancer cells (184A1, MCF10A). Upon cancer cell incubation, the cell-adhesion protein P-selectin was found to re-distribute into circular nano-structures, consistent with accumulation into the membrane of protein-storing alpha-granules within the platelets. These changes were to a significantly lesser extent, if at all, found in platelets incubated with normal cells, or in platelets subject to soluble platelet activators. From these patterns, we developed a classification procedure, whereby platelets exposed to cancer cells, to non-cancer cells, soluble activators, as well as non-activated platelets all could be identified in an automatic, objective manner. We demonstrate that STED imaging, in contrast to electron and confocal microscopy, has the necessary spatial resolution and labelling efficiency to identify protein distribution patterns in platelets and can resolve how they specifically change upon different activations. Combined with image analyses of specific protein distribution patterns within the platelets, STED imaging can thus have a role in future platelet-based cancer diagnostics and therapeutic monitoring. The presented approach can also bring further clarity into fundamental mechanisms for cancer cell-platelet interactions, and into non-contact cell-to-cell interactions in general.

  • 15.
    Bergstrand, Jan
    et al.
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics.
    Xu, Lei
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics.
    Miao, Xinyan
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics.
    Li, Nailin
    Karolinska Institutet, Department of Medicine, Karolinska University Hospital, L7:03, SE-171 76 Stockholm, Sweden.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Franzén, Bo
    Karolinska Institutet, Department of Medicine, Karolinska University Hospital, L7:03, SE-171 76 Stockholm, Sweden.
    Auer, Gert
    Karolinska Institutet, Department of Medicine, Karolinska University Hospital, L7:03, SE-171 76 Stockholm, Sweden.
    Lomnytska, Marta
    Karolinska Institutet, Department of Medicine, Karolinska University Hospital, L7:03, SE-171 76 Stockholm, Sweden.
    Widengren, Jerker
    KTH, School of Engineering Sciences (SCI), Applied Physics, Quantum and Biophotonics.
    Super-resolution microscopy can identify specific protein distribution patterns in platelets incubated with cancer cellsManuscript (preprint) (Other academic)
    Abstract [en]

    Protein contents in platelets are frequently changed upon tumor development and metastasis. However, how cancer cells can influence protein-selective redistribution and release within platelets, thereby promoting tumor development, remains largely elusive. With fluorescence-based super-resolution stimulated emission depletion (STED) imaging we reveal how specific proteins, implicated in tumor progression and metastasis, re-distribute within platelets, when subject to soluble activators (thrombin, adenosine-diphosphate and thromboxaneA2), and when incubated with cancer (MCF-7, MDA-MB-231, EFO21) or non-cancer cells (184A1, MCF10A). Upon cancer cell incubation, the cell-adhesion protein P-selectin was found to re-distribute into circular nano-structures, consistent with accumulation into the membrane of protein-storing alpha-granules within the platelets. These changes were to a significantly lesser extent, if at all, found in platelets incubated with normal cells, or in platelets subject to soluble platelet activators. From these patterns, we developed a classification procedure, whereby platelets exposed to cancer cells, to non-cancer cells, soluble activators as well as non-activated platelets all could be identified in an automatic, objective manner. We demonstrate that STED imaging, in contrast to electron and confocal microscopy, has the necessary spatial resolution and labelling efficiency to identify protein distribution patterns in platelets and can resolve how they specifically change upon different activations. Combined with image analyses of specific protein distribution patterns within the platelets, STED imaging can thus have a role in future platelet-based cancer diagnostics and therapeutic monitoring. The presented approach can also bring further clarity into fundamental mechanisms for cancer cell-platelet interactions, and into non-contact cell-to-cell interactions in general. 

    Download full text (pdf)
    fulltext
  • 16.
    Buddenkotte, Thomas
    et al.
    Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom; Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Department for Diagnostic and Interventional Radiology and Nuclear Medicine, University Hospital Hamburg-Eppendorf, Hamburg, Germany; Jung diagnostics GmbH, Hamburg, Germany.
    Escudero Sanchez, Lorena
    Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom.
    Crispin-Ortuzar, Mireia
    Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Institute, University of Cambridge, Cambridge, United Kingdom; Department of Oncology, University of Cambridge, Cambridge, United Kingdom.
    Woitek, Ramona
    Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom; Medical Image Analysis & Artificial Intelligence (MIAAI), Department of Medicine, Danube Private University, Krems, Austria.
    McCague, Cathal
    Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom.
    Brenton, James D.
    Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Institute, University of Cambridge, Cambridge, United Kingdom; Department of Oncology, University of Cambridge, Cambridge, United Kingdom.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Numerical Analysis, NA.
    Sala, Evis
    Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom; Dipartimento di Scienze Radiologiche ed Ematologiche, Università Cattolica del Sacro Cuore, Rome, Italy; Dipartimento Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Policlinico Universitario A. Gemelli IRCCS, Rome, Italy.
    Rundo, Leonardo
    Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, United Kingdom; Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno, Fisciano (SA), Italy.
    Calibrating ensembles for scalable uncertainty quantification in deep learning-based medical image segmentation2023In: Computers in Biology and Medicine, ISSN 0010-4825, E-ISSN 1879-0534, Vol. 163, article id 107096Article in journal (Refereed)
    Abstract [en]

    Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, quantifying the uncertainty of the models can play a critical role for example in active learning or machine human interaction. Uncertainty quantification is especially difficult when using deep learning-based models, which are the state-of-the-art in many imaging applications. The current uncertainty quantification approaches do not scale well in high-dimensional real-world problems. Scalable solutions often rely on classical techniques, such as dropout, during inference or training ensembles of identical models with different random seeds to obtain a posterior distribution. In this paper, we present the following contributions. First, we show that the classical approaches fail to approximate the classification probability. Second, we propose a scalable and intuitive framework for uncertainty quantification in medical image segmentation that yields measurements that approximate the classification probability. Third, we suggest the usage of k-fold cross-validation to overcome the need for held out calibration data. Lastly, we motivate the adoption of our method in active learning, creating pseudo-labels to learn from unlabeled images and human–machine collaboration.

  • 17.
    Buddenkotte, Thomas
    et al.
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England.;Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Hosp Hamburg Eppendorf, Dept Diagnost & Intervent Radiol & Nucl Med, Hamburg, Germany.;Jung Diagnost GmbH, Hamburg, Germany..
    Rundo, Leonardo
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Univ Salerno, Dept Informat & Elect Engn & Appl Math, Fisciano, Italy..
    Woitek, Ramona
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Danube Private Univ, Dept Med, Krems, Austria..
    Sanchez, Lorena Escudero
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England..
    Beer, Lucian
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Med Univ Vienna, Dept Biomed Imaging & Image Guided Therapy, Vienna, Austria..
    Crispin-Ortuzar, Mireia
    Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Univ Cambridge, Canc Res UK Cambridge Inst, Cambridge, England.;Univ Cambridge, Dept Oncol, Cambridge, England..
    Etmann, Christian
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Mukherjee, Subhadip
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Bura, Vlad
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Cty Clin Emergency Hosp, Dept Radiol & Med Imaging, Cluj Napoca, Romania..
    McCague, Cathal
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England..
    Sahin, Hilal
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Tepecik Training & Res Hosp, Dept Radiol, Izmir, Turkiye..
    Pintican, Roxana
    Cty Clin Emergency Hosp, Dept Radiol & Med Imaging, Cluj Napoca, Romania.;Iuliu Hatieganu Univ Med & Pharm, Dept Radiol, Cluj Napoca 400012, Romania..
    Zerunian, Marta
    Sapienza Univ Rome, St Andrea Hosp, Dept Med Surg & Translat Med, Radiol Unit, Rome, Italy..
    Allajbeu, Iris
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England..
    Singh, Naveena
    Dept Clin Pathol, Barts Hlth NHS Trust, London, England..
    Sahdev, Anju
    Barts Hlth NHS Trust, Dept Radiol, London, England..
    Havrilesky, Laura
    Duke Univ, Med Ctr, Durham, NC USA..
    Cohn, David E.
    Ohio State Univ, Coll Med, Div Gynecol Oncol, Dept Obstet & Gynecol,Comprehens Canc Ctr, Columbus, OH USA..
    Bateman, Nicholas W.
    Uniformed Serv Univ Hlth Sci, Dept Obstet & Gynecol, Gynecol Canc Ctr Excellence, Walter Reed Natl Mil Med Ctr, Bethesda, MD USA.;Walter Reed Natl Mil Med Ctr, John P Murtha Canc Ctr, Bethesda, MD USA..
    Conrads, Thomas P.
    Uniformed Serv Univ Hlth Sci, Dept Obstet & Gynecol, Gynecol Canc Ctr Excellence, Walter Reed Natl Mil Med Ctr, Bethesda, MD USA.;Walter Reed Natl Mil Med Ctr, John P Murtha Canc Ctr, Bethesda, MD USA.;Dept Obstet & Gynecol, Inova Fairfax Med Campus, Falls Church, VA USA.;Inova Ctr Personalized Hlth, Inova Schar Canc Inst, Falls Church, VA USA..
    Darcy, Kathleen M.
    Uniformed Serv Univ Hlth Sci, Dept Obstet & Gynecol, Gynecol Canc Ctr Excellence, Walter Reed Natl Mil Med Ctr, Bethesda, MD USA.;Walter Reed Natl Mil Med Ctr, John P Murtha Canc Ctr, Bethesda, MD USA..
    Maxwell, G. Larry
    Uniformed Serv Univ Hlth Sci, Dept Obstet & Gynecol, Gynecol Canc Ctr Excellence, Walter Reed Natl Mil Med Ctr, Bethesda, MD USA.;Walter Reed Natl Mil Med Ctr, John P Murtha Canc Ctr, Bethesda, MD USA.;Dept Obstet & Gynecol, Inova Fairfax Med Campus, Falls Church, VA USA..
    Freymann, John B.
    Frederick Natl Lab Canc Res, Canc Imaging Informat Lab, Frederick, MD USA..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Brenton, James D.
    Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Univ Cambridge, Canc Res UK Cambridge Inst, Cambridge, England..
    Sala, Evis
    Univ Cambridge, Dept Radiol, Box 218,Cambridge Biomed Campus, Cambridge CB2 0QQ, England.;Univ Cambridge, Canc Res UK Cambridge Ctr, Cambridge, England.;Univ Cattolica Sacro Cuore, Dipartimento Sci Radiol & Ematol, Rome, Italy.;Policlin Univ A Gemelli IRCCS, Dipartimento Diagnost Immagini Radioterapia Oncol, Rome, Italy..
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Deep learning-based segmentation of multisite disease in ovarian cancer2023In: EUROPEAN RADIOLOGY EXPERIMENTAL, ISSN 2509-9280, Vol. 7, no 1, article id 77Article in journal (Refereed)
    Abstract [en]

    Purpose: To determine if pelvic/ovarian and omental lesions of ovarian cancer can be reliably segmented on computed tomography (CT) using fully automated deep learning-based methods.

    Methods: A deep learning model for the two most common disease sites of high-grade serous ovarian cancer lesions (pelvis/ovaries and omentum) was developed and compared against the well-established “no-new-Net” framework and unrevised trainee radiologist segmentations. A total of 451 CT scans collected from four different institutions were used for training (n = 276), evaluation (n = 104) and testing (n = 71) of the methods. The performance was evaluated using the Dice similarity coefficient (DSC) and compared using a Wilcoxon test.

    Results: Our model outperformed no-new-Net for the pelvic/ovarian lesions in cross-validation, on the evaluation and test set by a significant margin (p values being 4 × 10–7, 3 × 10–4, 4 × 10–2, respectively), and for the omental lesions on the evaluation set (p = 1 × 10–3). Our model did not perform significantly differently in segmenting pelvic/ovarian lesions (p = 0.371) compared to a trainee radiologist. On an independent test set, the model achieved a DSC performance of 71 ± 20 (mean ± standard deviation) for pelvic/ovarian and 61 ± 24 for omental lesions.

    Conclusion: Automated ovarian cancer segmentation on CT scans using deep neural networks is feasible and achieves performance close to a trainee-level radiologist for pelvic/ovarian lesions.

    Relevance statement: Automated segmentation of ovarian cancer may be used by clinicians for CT-based volumetric assessments and researchers for building complex analysis pipelines.

    Key points:

    • The first automated approach for pelvic/ovarian and omental ovarian cancer lesion segmentation on CT images has been presented.
    • Automated segmentation of ovarian cancer lesions can be comparable with manual segmentation of trainee radiologists.
    • Careful hyperparameter tuning can provide models significantly outperforming strong state-of-the-art baselines. Graphical Abstract: [Figure not available: see fulltext.]
  • 18. Chen, C.
    et al.
    Gris, B.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    A new variational model for joint image reconstruction and motion estimation in spatiotemporal imaging2019In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 12, no 4, p. 1686-1719Article in journal (Refereed)
    Abstract [en]

    We propose a new variational model for joint image reconstruction and motion estimation in spatiotemporal imaging, which is investigated along a general framework that we present with shape theory. This model consists of two components, one for conducting modified static image reconstruction, and the other performs sequentially indirect image registration. For the latter, we generalize the large deformation diffeomorphic metric mapping framework into the sequentially indirect registration setting. The proposed model is compared theoretically against alternative approaches (optical flow based model and diffeomorphic motion models), and we demonstrate that the proposed model has desirable properties in terms of the optimal solution. The theoretical derivations and efficient algorithms are also presented for a time-discretized scenario of the proposed model, which show that the optimal solution of the time-discretized version is consistent with that of the time-continuous one, and most of the computational components is the easy-implemented linearized deformation. The complexity of the algorithm is analyzed as well. This work is concluded by some numerical examples in 2D space + time tomography with very sparse and/or highly noisy data.

  • 19. Chen, C.
    et al.
    Wang, R.
    Bajaj, C.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    An efficient algorithm to compute the X-ray transform2021In: International Journal of Computer Mathematics, ISSN 0020-7160, E-ISSN 1029-0265Article in journal (Refereed)
    Abstract [en]

    We propose a new algorithm to compute the X-ray transform of an image represented by unit (pixel/voxel) basis functions. The fundamental task is equivalently calculating the intersection lengths of the ray with associated units. For the given ray, we derive the sufficient and necessary condition for non-vanishing intersectability. By this condition, we can distinguish the units that produce valid intersections with the ray. Only for those units, we calculate the intersection lengths by the obtained analytic formula. The proposed algorithm is adapted to various two-dimensional (2D)/three-dimensional (3D) scanning geometries, and its several issues are also discussed, including the intrinsic ambiguity, flexibility, computational cost and parallelization. The proposed method is fast and easy to implement, more complete and flexible than the existing alternatives with respect to different scanning geometries and different basis functions. Finally, we validate the correctness of the algorithm.

  • 20. Chen, C.
    et al.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Indirect image registration with large diffeomorphic deformations2018In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 11, no 1, p. 575-617Article in journal (Refereed)
    Abstract [en]

    This paper adapts the large deformation diffeomorphic metric mapping framework for image registration to the indirect setting, where a template is registered against a target that is given through indirect noisy observations. The registration uses diffeomorphisms that transform the template through a (group) action. These diffeomorphisms are generated by solving a flow equation that is defined by a velocity field with certain regularity. The theoretical analysis includes a proof that indirect image registration has solutions (existence) that are stable and that converge as the data error tends to zero, so it becomes a well-defined regularization method. The paper concludes with examples of indirect image registration in 2D tomography with very sparse and/or highly noisy data. 

  • 21.
    Diepeveen, Willem
    et al.
    Faculty of Mathematics, University of Cambridge, Cambridge, England.
    Lellmann, Jan
    Institute of Mathematics and Image Computing, University of Lubeck, Lubeck, Germany.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Numerical Analysis, NA.
    Schonlieb, Carola Bibiane
    Faculty of Mathematics, University of Cambridge, Cambridge, England.
    Regularizing Orientation Estimation in Cryogenic Electron Microscopy Three-Dimensional Map Refinement through Measure-Based Lifting over Riemannian Manifolds2023In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 16, no 3, p. 1440-1490Article in journal (Refereed)
    Abstract [en]

    Motivated by the trade-off between noise robustness and data consistency for joint three-imensional (3D) map reconstruction and rotation estimation in single particle cryogenic-electron microscopy (Cryo-EM), we propose ellipsoidal support lifting (ESL), a measure-based lifting scheme for regularizing and approximating the global minimizer of a smooth function over a Riemannian manifold. Under a uniqueness assumption on the minimizer we show several theoretical results, in particular well-posedness of the method and an error bound due to the induced bias with respect to the global minimizer. Additionally, we use the developed theory to integrate the measure-based lifting scheme into an alternating update method for joint homogeneous 3D map reconstruction and rotation estimation, where typically tens of thousands of manifold-valued minimization problems have to be solved and where regularization is necessary because of the high noise levels in the data. The joint recovery method is used to test both the theoretical predictions and algorithmic performance through numerical experiments with Cryo-EM data. In particular, the induced bias due to the regularizing effect of ESL empirically estimates better rotations, i.e., rotations closer to the ground truth, than global optimization would.

  • 22. Dong, G.
    et al.
    Patrone, A. R.
    Scherzer, O.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Infinite dimensional optimization models and PDEs for dejittering2015In: 5th International Conference on Scale Space and Variational Methods in Computer Vision, SSVM 2015, Elsevier, 2015, Vol. 9087, p. 678-689Conference paper (Refereed)
    Abstract [en]

    In this paper we do a systematic investigation of continuous methods for pixel, line pixel and line dejittering. The basis for these investigations are the discrete line dejittering algorithm of Nikolova and the partial differential equation of Lenzen et al for pixel dejittering. To put these two different worlds in perspective we find infinite dimensional optimization algorithms linking to the finite dimensional optimization problems and formal flows associated with the infinite dimensional optimization problems. Two different kinds of optimization problems will be considered: Dejittering algorithms for determining the displacement and displacement error correction formulations, which correct the jittered image, without estimating the jitter. As a by-product we find novel variational methods for displacement error regularization and unify them into one family. The second novelty is a comprehensive comparison of the different models for different types of jitter, in terms of efficiency of reconstruction and numerical complexity.

  • 23.
    Eguizabal, Alma
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Persson, Mats
    KTH, School of Engineering Sciences (SCI), Physics, Physics of Medical Imaging.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Learned Material Decomposition for Photon Counting CT2021In: Proceedings of the 16th Virtual International Meeting onFully 3D Image Reconstruction inRadiology and Nuclear Medicine, 2021, p. 15-19Conference paper (Other academic)
  • 24.
    Eguizabal, Alma
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Persson, Mats
    KTH, School of Engineering Sciences (SCI), Physics, Physics of Medical Imaging.
    A deep learning one-step solution to material image reconstruction in photon counting spectral CT2022In: Proceedings Volume 12031, Medical Imaging 2022: Physics of Medical Imaging, SPIE-Intl Soc Optical Eng , 2022Conference paper (Refereed)
  • 25.
    Esteve-Yague, Carlos
    et al.
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Diepeveen, Willem
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Spectral decomposition of atomic structures in heterogeneous cryo-EM2023In: Inverse Problems, ISSN 0266-5611, E-ISSN 1361-6420, Vol. 39, no 3, p. 034003-, article id 034003Article in journal (Refereed)
    Abstract [en]

    We consider the problem of recovering the three-dimensional atomic structure of a flexible macromolecule from a heterogeneous cryogenic electron microscopy (cryo-EM) dataset. The dataset contains noisy tomographic projections of the electrostatic potential of the macromolecule, taken from different viewing directions, and in the heterogeneous case, each cryo-EM image corresponds to a different conformation of the macromolecule. Under the assumption that the macromolecule can be modelled as a chain, or discrete curve (as it is for instance the case for a protein backbone with a single chain of amino-acids), we introduce a method to estimate the deformation of the atomic model with respect to a given conformation, which is assumed to be known a priori. Our method consists on estimating the torsion and bond angles of the atomic model in each conformation as a linear combination of the eigenfunctions of the Laplace operator in the manifold of conformations. These eigenfunctions can be approximated by means of a well-known technique in manifold learning, based on the construction of a graph Laplacian using the cryo-EM dataset. Finally, we test our approach with synthetic datasets, for which we recover the atomic model of two-dimensional and three-dimensional flexible structures from simulated cryo-EM images.

  • 26. Gopinath, A.
    et al.
    Xu, G.
    Ress, D.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.). KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM.
    Subramaniam, S.
    Bajaj, C.
    Shape-based regularization of electron tomographic reconstruction2012In: IEEE Transactions on Medical Imaging, ISSN 0278-0062, E-ISSN 1558-254X, Vol. 31, no 12, p. 2241-2252Article in journal (Refereed)
    Abstract [en]

    We introduce a tomographic reconstruction method implemented using a shape-based regularization technique. Spatial models of known features in the structure being reconstructed are integrated into the reconstruction process as regularizers. Our regularization scheme is driven locally through shape information obtained from segmentation and compared with a known spatial model. We demonstrated our method on tomography data from digital phantoms, simulated data, and experimental electron tomography (ET) data of virus complexes. Our reconstruction showed reduced blurring and an improvement in the resolution of the reconstructed volume was also measured. This method also produced improved demarcation of spike boundaries in viral membranes when compared with popular techniques like weighted back projection and the algebraic reconstruction technique. Improved ET reconstructions will provide better structure elucidation and improved feature visualization, which can aid in solving key biological issues. Our method can also be generalized to other tomographic modalities.

  • 27.
    Gris, Barbara
    et al.
    Univ Paris, LJLL, Sorbonne Univ, CNRS, F-75005 Paris, France..
    Chen, Chong
    Chinese Acad Sci, Acad Math & Syst Sci, ICMSEC, LSEC, Beijing 100190, Peoples R China..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Image reconstruction through metamorphosis2020In: Inverse Problems, ISSN 0266-5611, E-ISSN 1361-6420, Vol. 36, no 2, article id 025001Article in journal (Refereed)
    Abstract [en]

    The paper describes a method for reconstructing an image from noisy and indirect observations by registering, via metamorphosis, a template. The image registration part consists of two components, one is a geometric deformation that moves intensities without changing them and the other that changes intensity values. Unlike a registration with only geometrical deformation, this framework gives good results also when intensities of the template are poorly chosen. It also allows for appearance of a new structure. The approach is applicable to general inverse problems in imaging and we prove existence, stability and convergence, which implies that the method is a well-defined regularisation method. We also present several numerical examples from tomography.

  • 28. Hahn, S.
    et al.
    Mueller, Y.
    Hofmann, R.
    Moosmann, Julian
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Helfen, L.
    Guigay, J. -P
    van de Kamp, Th
    Baumbach, T.
    Spectral transfer from phase to intensity in Fresnel diffraction2016In: PHYSICAL REVIEW A, ISSN 2469-9926, Vol. 93, no 5, article id 053834Article in journal (Refereed)
    Abstract [en]

    We analyze theoretically and investigate experimentally the transfer of phase to intensity power spectra of spatial frequencies through free-space Fresnel diffraction. Depending on lambda z (where lambda is the wavelength and z is the free-space propagation distance) and the phase-modulation strength S, we demonstrate that for multiscale and broad phase spectra critical behavior transmutes a quasilinear to a nonlinear diffractogram except for low frequencies. On the contrary, a single-scale and broad phase spectrum induces a critical transition in the diffractogram at low frequencies. In both cases, identifying critical behavior encoded in the intensity power spectra is of fundamental interest because it exhibits the limits of perturbative power counting but also guides resolution and contrast optimization in propagation-based, single-distance, phase-contrast imaging, given certain dose and coherence constraints.

  • 29.
    Hauptmann, Andreas
    et al.
    Univ Oulu, Res Unit Math Sci, FIN-90570 Oulu, Finland.;UCL, Dept Comp Sci, London W E 6BT, England..
    Adler, Jonas
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). Elekta, S-11357 Stockholm, Sweden.;DeepMind, London N1C 4AG, England..
    Arridge, Simon
    UCL, Dept Comp Sci, London W E 6BT, England..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Multi-Scale Learned Iterative Reconstruction2020In: IEEE Transactions on Computational Imaging, ISSN 2573-0436, E-ISSN 2333-9403, Vol. 6, p. 843-856Article in journal (Refereed)
    Abstract [en]

    Model-based learned iterative reconstruction methods have recently been shown to outperform classical reconstruction algorithms. Applicability of these methods to large scale inverse problems is however limited by the available memory for training and extensive training times, the latter due to computationally expensive forward models. As a possible solution to these restrictions we propose a multi-scale learned iterative reconstruction scheme that computes iterates on discretisations of increasing resolution. This procedure does not only reduce memory requirements, it also considerably speeds up reconstruction and training times, but most importantly is scalable to large scale inverse problems with non-trivial forward operators, such as those that arise in many 3D tomographic applications. In particular, we propose a hybrid network that combines the multi-scale iterative approach with a particularly expressive network architecture which in combination exhibits excellent scalability in 3D. Applicability of the algorithm is demonstrated for 3D cone beam computed tomography from real measurement data of an organic phantom. Additionally, we examine scalability and reconstruction quality in comparison to established learned reconstruction methods in two dimensions for low dose computed tomography on human phantoms.

  • 30.
    Kimanius, Dari
    et al.
    MRC Lab Mol Biol, Cambridge, England..
    Zickert, Gustav
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Nakane, Takanori
    MRC Lab Mol Biol, Cambridge, England..
    Adler, Jonas
    DeepMind, London, England..
    Lunz, Sebastian
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Dept Appl Math & Theoret Phys, Cambridge, England..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Scheres, Sjors H. W.
    MRC Lab Mol Biol, Cambridge, England..
    Exploiting prior knowledge about biological macromolecules in cryo-EM structure determination2021In: IUCrJ, E-ISSN 2052-2525, Vol. 8, p. 60-75Article in journal (Refereed)
    Abstract [en]

    Three-dimensional reconstruction of the electron-scattering potential of biological macromolecules from electron cryo-microscopy (cryo-EM) projection images is an ill-posed problem. The most popular cryo-EM software solutions to date rely on a regularization approach that is based on the prior assumption that the scattering potential varies smoothly over three-dimensional space. Although this approach has been hugely successful in recent years, the amount of prior knowledge that it exploits compares unfavorably with the knowledge about biological structures that has been accumulated over decades of research in structural biology. Here, a regularization framework for cryo-EM structure determination is presented that exploits prior knowledge about biological structures through a convolutional neural network that is trained on known macromolecular structures. This neural network is inserted into the iterative cryo-EM structure-determination process through an approach that is inspired by regularization by denoising. It is shown that the new regularization approach yields better reconstructions than the current state of the art for simulated data, and options to extend this work for application to experimental cryo-EM data are discussed.

  • 31. Lang, L. F.
    et al.
    Neumayer, S.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM. KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Schönlieb, C. -B
    Template-Based Image Reconstruction from Sparse Tomographic Data2019In: Applied mathematics and optimization, ISSN 0095-4616, E-ISSN 1432-0606Article in journal (Refereed)
    Abstract [en]

    We propose a variational regularisation approach for the problem of template-based image reconstruction from indirect, noisy measurements as given, for instance, in X-ray computed tomography. An image is reconstructed from such measurements by deforming a given template image. The image registration is directly incorporated into the variational regularisation approach in the form of a partial differential equation that models the registration as either mass- or intensity-preserving transport from the template to the unknown reconstruction. We provide theoretical results for the proposed variational regularisation for both cases. In particular, we prove existence of a minimiser, stability with respect to the data, and convergence for vanishing noise when either of the abovementioned equations is imposed and more general distance functions are used. Numerically, we solve the problem by extending existing Lagrangian methods and propose a multilevel approach that is applicable whenever a suitable downsampling procedure for the operator and the measured data can be provided. Finally, we demonstrate the performance of our method for template-based image reconstruction from highly undersampled and noisy Radon transform data. We compare results for mass- and intensity-preserving image registration, various regularisation functionals, and different distance functions. Our results show that very reasonable reconstructions can be obtained when only few measurements are available and demonstrate that the use of a normalised cross correlation-based distance is advantageous when the image intensities between the template and the unknown image differ substantially.

  • 32. Lunz, S.
    et al.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM. KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Schönlieb, C. -B
    Adversarial regularizers in inverse problems2018In: Advances in Neural Information Processing Systems, Neural information processing systems foundation , 2018, p. 8507-8516Conference paper (Refereed)
    Abstract [en]

    Inverse Problems in medical imaging and computer vision are traditionally solved using purely model-based methods. Among those variational regularization models are one of the most popular approaches. We propose a new framework for applying data-driven approaches to inverse problems, using a neural network as a regularization functional. The network learns to discriminate between the distribution of ground truth images and the distribution of unregularized reconstructions. Once trained, the network is applied to the inverse problem by solving the corresponding variational problem. Unlike other data-based approaches for inverse problems, the algorithm can be applied even if only unsupervised training data is available. Experiments demonstrate the potential of the framework for denoising on the BSDS dataset and for computed tomography reconstruction on the LIDC dataset.

  • 33.
    Mukherjee, S.
    et al.
    IIT-Kharagpur, India.
    Dittmer, S.
    University of Cambridge, UK.
    Shumaylov, Z.
    University of Cambridge, UK.
    Lunz, S.
    University of Cambridge, UK.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Schönlieb, C. B.
    University of Cambridge, UK.
    DATA-DRIVEN CONVEX REGULARIZERS FOR INVERSE PROBLEMS2024In: 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024 - Proceedings, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 13386-13390Conference paper (Refereed)
    Abstract [en]

    We propose to learn a data-adaptive convex regularizer, which is parameterized using an input-convex neural network (ICNN), for variational image reconstruction. The regularizer parameters are learned adversarially by telling apart clean images from the artifact-ridden ones in a training dataset. Convexity of the regularizer is theoretically and practically important since (i) one can establish well-posedness guarantees for the corresponding variational reconstruction problem and (ii) devise provably convergent optimization algorithms for reconstruction. In particular, the resulting method is shown to be convergent in the sense of regularization and can be solved provably using a gradient-based solver. To demonstrate the performance of our approach for solving inverse problems, we consider deblurring natural images and reconstruction in X-ray computed tomography (CT) and show that the proposed convex regularizer is on par with and sometimes superior to state-of-the-art classical and data-driven techniques for inverse problems, especially with severely ill-posed forward operators (such as in limited-angle tomography).

  • 34. Mukherjee, S.
    et al.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Schönlieb, C. -B
    Adversarially Learned Iterative Reconstruction for Imaging Inverse Problems2021In: 8th International Conference on Scale Space and Variational Methods in Computer Vision, SSVM 2021, Springer Science and Business Media Deutschland GmbH , 2021, p. 540-552Conference paper (Refereed)
    Abstract [en]

    In numerous practical applications, especially in medical image reconstruction, it is often infeasible to obtain a large ensemble of ground-truth/measurement pairs for supervised learning. Therefore, it is imperative to develop unsupervised learning protocols that are competitive with supervised approaches in performance. Motivated by the maximum-likelihood principle, we propose an unsupervised learning framework for solving ill-posed inverse problems. Instead of seeking pixel-wise proximity between the reconstructed and the ground-truth images, the proposed approach learns an iterative reconstruction network whose output matches the ground-truth in distribution. Considering tomographic reconstruction as an application, we demonstrate that the proposed unsupervised approach not only performs on par with its supervised variant in terms of objective quality measures, but also successfully circumvents the issue of over-smoothing that supervised approaches tend to suffer from. The improvement in reconstruction quality comes at the expense of higher training complexity, but, once trained, the reconstruction time remains the same as its supervised counterpart. 

  • 35. Mukherjee, Subhadip
    et al.
    Carioni, M.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Schönlieb, C. -B
    End-to-end reconstruction meets data-driven regularization for inverse problems2021In: Advances in Neural Information Processing Systems, Neural information processing systems foundation , 2021, p. 21413-21425Conference paper (Refereed)
    Abstract [en]

    We propose a new approach for learning end-to-end reconstruction operators based on unpaired training data for ill-posed inverse problems. The proposed method combines the classical variational framework with iterative unrolling and essentially seeks to minimize a weighted combination of the expected distortion in the measurement space and the Wasserstein-1 distance between the distributions of the reconstruction and the ground-truth. More specifically, the regularizer in the variational setting is parametrized by a deep neural network and learned simultaneously with the unrolled reconstruction operator. The variational problem is then initialized with the output of the reconstruction network and solved iteratively till convergence. Notably, it takes significantly fewer iterations to converge as compared to variational methods, thanks to the excellent initialization obtained via the unrolled operator. The resulting approach combines the computational efficiency of end-to-end unrolled reconstruction with the well-posedness and noise-stability guarantees of the variational setting. Moreover, we demonstrate with the example of image reconstruction in X-ray computed tomography (CT) that our approach outperforms state-of-the-art unsupervised methods and that it outperforms or is at least on par with state-of-the-art supervised data-driven reconstruction approaches.

  • 36.
    Mukherjee, Subhadip
    et al.
    Univ Bath, Dept Comp Sci, Machine Learning & Artificial Intelligence, Bath BA2 7PB, England..
    Hauptmann, Andreas
    Univ Oulu, Computat Math Res Unit Math Sci, Oulu 90014, Finland.;UCL, Dept Comp Sci, London W E 6BT, England..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Numerical Analysis, NA. Uppsala Univ, Dept Informat Technol, Computat Sci, S-75237 Uppsala, Sweden..
    Pereyra, Marcelo
    Heriot Watt Univ, Maxwell Inst Math Sci, Edinburgh EH14 4AS, Scotland.;Heriot Watt Univ, Sch Math & Comp Sci, Edinburgh EH14 4AS, Scotland..
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Appl Math, Cambridge CB3 0WA, England.;Cambridge Image Anal Grp, Cambridge, England.;EPSRC Cambridge Math Informat Healthcare Hub, Cambridge, England..
    Learned Reconstruction Methods With Convergence Guarantees: A survey of concepts and applications2023In: IEEE signal processing magazine (Print), ISSN 1053-5888, E-ISSN 1558-0792, Vol. 40, no 1, p. 164-182Article in journal (Refereed)
    Abstract [en]

    In recent years, deep learning has achieved remarkable empirical success for image reconstruction. This has catalyzed an ongoing quest for the precise characterization of the correctness and reliability of data-driven methods in critical use cases, for instance, in medical imaging. Notwithstanding the excellent performance and efficacy of deep learning-based methods, concerns have been raised regarding the approaches' stability, or lack thereof, with serious practical implications. Significant advances have been made in recent years to unravel the inner workings of data-driven image recovery methods, challenging their widely perceived black-box nature. In this article, we specify relevant notions of convergence for data-driven image reconstruction, which forms the basis of a survey of learned methods with mathematically rigorous reconstruction guarantees. An example that is highlighted is the role of input-convex neural networks (ICNNs), offering the possibility to combine the power of deep learning with classical convex regularization theory for devising methods that are provably convergent. This survey article is aimed at both methodological researchers seeking to advance the frontiers of our understanding of data-driven image reconstruction methods as well as practitioners by providing an accessible description of useful convergence concepts and by placing some of the existing empirical practices on a solid mathematical foundation.

  • 37. Norlén, L.
    et al.
    Anwar, J.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Accessing the molecular organization of the stratum corneum using high-resolution electron microscopy and computer simulation2014In: Computational Biophysics of the Skin, Pan Stanford Publishing, 2014, p. 289-330Chapter in book (Other academic)
  • 38. Norlén, L.
    et al.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM. KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Skoglund, U.
    Molecular cryo-electron tomography of vitreous tissue sections: current challenges2009In: Journal of Microscopy, ISSN 0022-2720, E-ISSN 1365-2818, Vol. 235, no 3, p. 293-307Article in journal (Refereed)
    Abstract [en]

    Electron tomography of vitreous tissue sections (tissue TOVIS) allows the study of the three-dimensional structure of molecular complexes in a near-native cellular context. Its usage is, however, limited by an unfortunate combination of noisy and incomplete data, by a technically demanding sample preparation procedure, and by a disposition for specimen degradation during data collection. Here we outline some major challenges as experienced from the application of TOVIS to human skin. We further consider a number of practical measures as well as theoretical approaches for its future development.

  • 39. Quinto, Eric Todd
    et al.
    Ozan, Öktem
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Skoglund, Ulf
    Reply to Wang and Yu: Both electron lambda tomography and interior tomography have their uses2010In: Proceedings of the National Academy of Sciences of the United States of America, ISSN 0027-8424, E-ISSN 1091-6490, Vol. 107, no 22, p. E94-E95Article in journal (Other academic)
  • 40.
    Reuss, Matthias
    et al.
    KTH, School of Engineering Sciences (SCI), Applied Physics. KTH, Centres, Science for Life Laboratory, SciLifeLab.
    Förds, F.
    Blom, Hans
    KTH, School of Engineering Sciences (SCI), Applied Physics, Cellular Biophysics. KTH, Centres, Science for Life Laboratory, SciLifeLab.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Högberg, B.
    Brismar, Hans
    KTH, School of Engineering Sciences (SCI), Applied Physics, Cellular Biophysics. KTH, Centres, Science for Life Laboratory, SciLifeLab. Karolinska Institutet, Sweden.
    Measuring true localization accuracy in super resolution microscopy with DNA-origami nanostructures2017In: New Journal of Physics, E-ISSN 1367-2630, Vol. 19, no 2, article id 025013Article in journal (Refereed)
    Abstract [en]

    A common method to assess the performance of (super resolution) microscopes is to use the localization precision of emitters as an estimate for the achieved resolution. Naturally, this is widely used in super resolution methods based on single molecule stochastic switching. This concept suffers from the fact that it is hard to calibrate measures against a real sample (a phantom), because true absolute positions of emitters are almost always unknown. For this reason, resolution estimates are potentially biased in an image since one is blind to true position accuracy, i.e. deviation in position measurement from true positions. We have solved this issue by imaging nanorods fabricated with DNA-origami. The nanorods used are designed to have emitters attached at each end in a well-defined and highly conserved distance. These structures are widely used to gauge localization precision. Here, we additionally determined the true achievable localization accuracy and compared this figure of merit to localization precision values for two common super resolution microscope methods STED and STORM.

  • 41.
    Ringh, Axel
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Optimization and Systems Theory.
    Zhuge, X.
    Palenstijn, W. J.
    Batenburg, K. J.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    High-level algorithm prototyping: An example extending the TVR-DART algorithm2017In: Discrete Geometry for Computer Imagery: 20th IAPR International Conference, DGCI 2017, Vienna, Austria, September 19 – 21, 2017, Proceedings, Springer, 2017, p. 109-121Chapter in book (Refereed)
    Abstract [en]

    Operator Discretization Library (ODL) is an open-source Python library for prototyping reconstruction methods for inverse problems, and ASTRA is a high-performance Matlab/Python toolbox for large-scale tomographic reconstruction. The paper demonstrates the feasibility of combining ODL with ASTRA to prototype complex reconstruction methods for discrete tomography. As a case in point, we consider the total-variation regularized discrete algebraic reconstruction technique (TVR-DART). TVR-DART assumes that the object to be imaged consists of a limited number of distinct materials. The ODL/ASTRA implementation of this algorithm makes use of standardized building blocks, that can be combined in a plug-and-play manner. Thus, this implementation of TVR-DART can easily be adapted to account for application specific aspects, such as various noise statistics that come with different imaging modalities.

  • 42.
    Rudzusika, Jevgenija
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Koehler, Thomas
    Philips Res, Hamburg, Germany..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Deep Learning-Based Dictionary Learning and Tomographic Image Reconstruction2022In: SIAM Journal on Imaging Sciences, E-ISSN 1936-4954, Vol. 15, no 4, p. 1729-1764Article in journal (Refereed)
    Abstract [en]

    This work presents an approach for image reconstruction in clinical low-dose tomography that combines principles from sparse signal processing with ideas from deep learning. First, we describe sparse signal representation in terms of dictionaries from a statistical perspective and interpret dictionary learning as a process of aligning the distribution that arises from a generative model with the empirical distribution of true signals. As a result, we can see that sparse coding with learned dictionaries resembles a specific variational autoencoder, where the encoder is a sparse coding algorithm and the decoder is a linear function. Next, we show that dictionary learning can also benefit from computational advancements introduced in the context of deep learning, such as parallelism and stochastic optimization. Finally, we show that regularization by dictionaries achieves competitive performance in computed tomography reconstruction compared to state-of-the-art model-based and data-driven approaches, while being unsupervised with respect to tomographic data.

  • 43. Rullgard, H.
    et al.
    Ofverstedt, L. -G
    Masich, S.
    Daneholt, B.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.). KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM.
    Simulation of transmission electron microscope images of biological specimens2011In: Journal of Microscopy, ISSN 0022-2720, E-ISSN 1365-2818, Vol. 243, no 3, p. 234-256Article in journal (Refereed)
    Abstract [en]

    We present a new approach to simulate electron cryo-microscope images of biological specimens. The framework for simulation consists of two parts; the first is a phantom generator that generates a model of a specimen suitable for simulation, the second is a transmission electron microscope simulator. The phantom generator calculates the scattering potential of an atomic structure in aqueous buffer and allows the user to define the distribution of molecules in the simulated image. The simulator includes a well defined electron-specimen interaction model based on the scalar Schrodinger equation, the contrast transfer function for optics, and a noise model that includes shot noise as well as detector noise including detector blurring. To enable optimal performance, the simulation framework also includes a calibration protocol for setting simulation parameters. To test the accuracy of the new framework for simulation, we compare simulated images to experimental images recorded of the Tobacco Mosaic Virus (TMV) in vitreous ice. The simulated and experimental images show good agreement with respect to contrast variations depending on dose and defocus. Furthermore, random fluctuations present in experimental and simulated images exhibit similar statistical properties. The simulator has been designed to provide a platform for development of new instrumentation and image processing procedures in single particle electron microscopy, two-dimensional crystallography and electron tomography with well documented protocols and an open source code into which new improvements and extensions are easily incorporated.

  • 44.
    Sanchez, Lorena Escudero
    et al.
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Li Ka Shing Ctr, Canc Res UK Cambridge Ctr, Cambridge CB2 0RE, England.;Natl Canc Imaging Translat Accelerator NCITA Conso, London, England..
    Buddenkotte, Thomas
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Univ Cambridge, Dept Appl Math & Theoret Phys, Wilberforce Rd, Cambridge CB3 0WA, England.;Univ Hosp Hamburg Eppendorf, Dept Diagnost & Intervent Radiol & Nucl Med, D-20246 Hamburg, Germany.;Jung Diagnost GmbH, D-22335 Hamburg, Germany..
    Al Sa'd, Mohammad
    Natl Canc Imaging Translat Accelerator NCITA Conso, London, England.;Imperial Coll, Canc Imaging Ctr, Dept Surg & Canc, London SW7 2AZ, England..
    McCague, Cathal
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Li Ka Shing Ctr, Canc Res UK Cambridge Ctr, Cambridge CB2 0RE, England.;Cambridge Univ Hosp NHS Fdn Trust, Cambridge CB2 0QQ, England..
    Darcy, James
    Natl Canc Imaging Translat Accelerator NCITA Conso, London, England.;Inst Canc Res, Div Radiotherapy & Imaging, London SW7 3RP, England..
    Rundo, Leonardo
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Li Ka Shing Ctr, Canc Res UK Cambridge Ctr, Cambridge CB2 0RE, England.;Univ Salerno, Dept Informat & Elect Engn & Appl Math DIEM, I-84084 Fisciano, Italy..
    Samoshkin, Alex
    Univ Cambridge, Sch Clin Med, Off Translat Res, Cambridge CB2 0SP, England..
    Graves, Martin J.
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Cambridge Univ Hosp NHS Fdn Trust, Cambridge CB2 0QQ, England..
    Hollamby, Victoria
    Univ Cambridge, Sch Clin Med, Res & Informat Governance, Cambridge CB2 0SP, England..
    Browne, Paul
    Univ Cambridge, High Performance Comp Dept, Cambridge CB3 0RB, England..
    Crispin-Ortuzar, Mireia
    Li Ka Shing Ctr, Canc Res UK Cambridge Ctr, Cambridge CB2 0RE, England.;Univ Cambridge, Dept Oncol, Cambridge CB2 0XZ, England..
    Woitek, Ramona
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Li Ka Shing Ctr, Canc Res UK Cambridge Ctr, Cambridge CB2 0RE, England.;Danube Private Univ, Fac Med & Dent, Res Ctr Med Image Anal & Artificial Intelligence M, Dept Med, A-3500 Krems, Austria..
    Sala, Evis
    Univ Cambridge, Dept Radiol, Cambridge CB2 0QQ, England.;Li Ka Shing Ctr, Canc Res UK Cambridge Ctr, Cambridge CB2 0RE, England.;Natl Canc Imaging Translat Accelerator NCITA Conso, London, England.;Cambridge Univ Hosp NHS Fdn Trust, Cambridge CB2 0QQ, England.;Univ Cattolica Sacro Cuore, Dipartimento Sci Radiol Ematol, I-00168 Rome, Italy.;Policlin Univ A Gemelli IRCCS, Dipartimento Diagnost Immagini Radioterapia Oncol, I-00168 Rome, Italy..
    Schonlieb, Carola-Bibiane
    Univ Cambridge, Dept Appl Math & Theoret Phys, Wilberforce Rd, Cambridge CB3 0WA, England..
    Doran, Simon J.
    Natl Canc Imaging Translat Accelerator NCITA Conso, London, England.;Inst Canc Res, Div Radiotherapy & Imaging, London SW7 3RP, England..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Integrating Artificial Intelligence Tools in the Clinical Research Setting: The Ovarian Cancer Use Case2023In: Diagnostics, ISSN 2075-4418, Vol. 13, no 17, article id 2813Article in journal (Refereed)
    Abstract [en]

    Artificial intelligence (AI) methods applied to healthcare problems have shown enormous potential to alleviate the burden of health services worldwide and to improve the accuracy and reproducibility of predictions. In particular, developments in computer vision are creating a paradigm shift in the analysis of radiological images, where AI tools are already capable of automatically detecting and precisely delineating tumours. However, such tools are generally developed in technical departments that continue to be siloed from where the real benefit would be achieved with their usage. Significant effort still needs to be made to make these advancements available, first in academic clinical research and ultimately in the clinical setting. In this paper, we demonstrate a prototype pipeline based entirely on open-source software and free of cost to bridge this gap, simplifying the integration of tools and models developed within the AI community into the clinical research setting, ensuring an accessible platform with visualisation applications that allow end-users such as radiologists to view and interact with the outcome of these AI tools.

  • 45.
    Siadat, Medya
    et al.
    Azarbaijan Shahid Madani Univ, Dept Appl Math, Tabriz 5375171379, Iran..
    Aghazadeh, Nasser
    Azarbaijan Shahid Madani Univ, Dept Appl Math, Tabriz 5375171379, Iran..
    Akbarifard, Farideh
    Azarbaijan Shahid Madani Univ, Dept Appl Math, Tabriz 5375171379, Iran..
    Brismar, Hjalmar
    KTH, School of Engineering Sciences (SCI), Applied Physics, Biophysics. KTH, Centres, Science for Life Laboratory, SciLifeLab. SciLifeLab, Adv Light Microscopy Facil, S-17165 Solna, Sweden..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). KTH, Centres, Science for Life Laboratory, SciLifeLab.
    Joint Image Deconvolution and Separation Using Mixed Dictionaries2019In: IEEE Transactions on Image Processing, ISSN 1057-7149, E-ISSN 1941-0042, Vol. 28, no 8, p. 3936-3945Article in journal (Refereed)
    Abstract [en]

    The task of separating an image into distinct components that represent different features plays an important role in many applications. Traditionally, such separation techniques are applied once the image in question has been reconstructed from measured data. We propose an efficient iterative algorithm, where reconstruction is performed jointly with the task of separation. A key assumption is that the image components have different sparse representations. The algorithm is based on a scheme that minimizes a functional composed of a data discrepancy term and the l(1)-norm of the coefficients of the different components with respect to their corresponding dictionaries. The performance is demonstrated for joint 2D deconvolution and separation into curve- and point-like components, and tests are performed on synthetic data as well as experimental stimulated emission depletion and confocal microscopy data. Experiments show that such a joint approach outperforms a sequential approach, where one first deconvolves data and then applies image separation.

  • 46.
    Siadat, Medya
    et al.
    Azarbaijan Shahid Madani Univ, Dept Appl Math, Tabriz, Iran..
    Aghazadeh, Nasser
    Azarbaijan Shahid Madani Univ, Image Proc Lab, Tabriz, Iran..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Reordering for improving global Arnoldi-Tikhonov method in image restoration problems2018In: Signal, Image and Video Processing, ISSN 1863-1703, E-ISSN 1863-1711, Vol. 12, no 3, p. 497-504Article in journal (Refereed)
    Abstract [en]

    This paper discusses the solution of large-scale linear discrete ill-posed problems arising from image restoration problems. Since the scale of the problem is usually very large, the computations with the blurring matrix can be very expensive. In this regard, we consider problems in which the coefficient matrix is the sum of Kronecker products of matrices to benefit the computation. Here, we present an alternative approach based on reordering of the image approximations obtained with the global Arnoldi-Tikhonov method. The ordering of the intensities is such that it makes the image approximation monotonic and thus minimizes the finite differences norm. We present theoretical properties of the method and numerical experiments on image restoration.

  • 47.
    Ström, Emanuel
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Persson, Mats
    KTH, School of Engineering Sciences (SCI), Physics, Physics of Medical Imaging. Karolinska Univ Hosp, BioClinicum, MedTech Labs, SE-17164 Solna, Sweden..
    Eguizabal, Alma
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.). KTH, School of Engineering Sciences (SCI), Physics.
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.). Uppsala Univ, Dept Informat Technol, S-75105 Uppsala, Sweden..
    Photon-Counting CT Reconstruction With a Learned Forward Operator2022In: IEEE Transactions on Computational Imaging, ISSN 2573-0436, E-ISSN 2333-9403, Vol. 8, p. 536-550Article in journal (Refereed)
    Abstract [en]

    Photon-Counting CT is an emerging imaging technology that promises higher spatial resolution and the possibility for material decomposition in the reconstruction. A major difficulty in Photon-Counting CT is to efficiently model cross-talk between detectors. In this work, we accelerate image reconstruction tasks for Photon-Counting CT by modelling the cross-talk with an appropriately trained deep convolutional neural network. The main result relates to proving convergence when using such a learned cross-talk model in the context of second-order optimisation methods for spectral CT. Another is to evaluate the method through numerical experiments on small-scale CT acquisitions generated using a realistic physics model. Using the reconstruction with a full cross-talk model as ground truth, the learned cross-talk model results in a 20 dB increase in peak-signal-to noise ratio compared to ignoring crass-talk altogether. At the same time, it effectively cuts the computation time of the full cross-talk model in half. Furthermore, the learned cross-talk model generalises well to both unseen data and unseen detector settings. Our results indicate that such a partially learned forward operator is a suitable way of modelling data generation in Photon-Counting CT with a computational benefit that becomes more noticeable for realistic problem sizes.

  • 48.
    Tavabi, Amir H.
    et al.
    Forschungszentrum Julich, Ernst Ruska Ctr Microscopy & Spect Electrons, D-52428 Julich, Germany.;Forschungszentrum Julich, Peter Grunberg Inst, D-52428 Julich, Germany..
    Beleggia, Marco
    Tech Univ Denmark, Ctr Elect Nanoscopy, DK-2800 Lyngby, Denmark..
    Migunov, Vadim
    Forschungszentrum Julich, Ernst Ruska Ctr Microscopy & Spect Electrons, D-52428 Julich, Germany.;Forschungszentrum Julich, Peter Grunberg Inst, D-52428 Julich, Germany..
    Savenko, Alexey
    FEI Co, Achtseweg Noord 5, NL-5600 KA Eindhoven, Netherlands..
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.). KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM.
    Dunin-Borkowski, Rafal E.
    Forschungszentrum Julich, Ernst Ruska Ctr Microscopy & Spect Electrons, D-52428 Julich, Germany.;Forschungszentrum Julich, Peter Grunberg Inst, D-52428 Julich, Germany..
    Pozzi, Giulio
    Forschungszentrum Julich, Ernst Ruska Ctr Microscopy & Spect Electrons, D-52428 Julich, Germany.;Forschungszentrum Julich, Peter Grunberg Inst, D-52428 Julich, Germany.;Univ Bologna, Dept Phys & Astron, Viale B Pichat 6-2, I-40127 Bologna, Italy..
    Tunable Ampere phase plate for low dose imaging of biomolecular complexes2018In: Scientific Reports, E-ISSN 2045-2322, Vol. 8, article id 5592Article in journal (Refereed)
    Abstract [en]

    A novel device that can be used as a tunable support-free phase plate for transmission electron microscopy of weakly scattering specimens is described. The device relies on the generation of a controlled phase shift by the magnetic field of a segment of current-carrying wire that is oriented parallel or antiparallel to the electron beam. The validity of the concept is established using both experimental electron holographic measurements and a theoretical model based on Ampere's law. Computer simulations are used to illustrate the resulting contrast enhancement for studies of biological cells and macromolecules.

  • 49. Vulovic, Milos
    et al.
    Ravelli, Raimond B. G.
    van Vliet, Lucas J.
    Koster, Abraham J.
    Lazic, Ivan
    Lucken, Uwe
    Rullgård, Hans
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.). KTH, School of Engineering Sciences (SCI), Centres, Center for Industrial and Applied Mathematics, CIAM.
    Rieger, Bernd
    Image formation modeling in cryo-electron microscopy2013In: Journal of Structural Biology, ISSN 1047-8477, E-ISSN 1095-8657, Vol. 183, no 1, p. 19-32Article in journal (Refereed)
    Abstract [en]

    Accurate modeling of image formation in cryo-electron microscopy is an important requirement for quantitative image interpretation and optimization of the data acquisition strategy. Here we present a forward model that accounts for the specimen's scattering properties, microscope optics, and detector response. The specimen interaction potential is calculated with the isolated atom superposition approximation (IASA) and extended with the influences of solvent's dielectric and ionic properties as well as the molecular electrostatic distribution. We account for an effective charge redistribution via the Poisson-Boltzmann approach and find that the IASA-based potential forms the dominant part of the interaction potential, as the contribution of the redistribution is less than 10%. The electron wave is propagated through the specimen by a multislice approach and the influence of the optics is included via the contrast transfer function. We incorporate the detective quantum efficiency of the camera due to the difference between signal and noise transfer characteristics, instead of using only the modulation transfer function. The full model was validated against experimental images of 20S proteasome, hemoglobin, and GroEL. The simulations adequately predict the effects of phase contrast, changes due to the integrated electron flux, thickness, inelastic scattering, detective quantum efficiency and acceleration voltage. We suggest that beam-induced specimen movements are relevant in the experiments whereas the influence of the solvent amorphousness can be neglected. All simulation parameters are based on physical principles and, when necessary, experimentally determined.

  • 50.
    Zickert, Gustav
    et al.
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematics (Div.).
    Öktem, Ozan
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.).
    Yarman, Can Evren
    Etud & Prod Schlumberger, 1 Rue Henri Becquerel, F-92140 Clamart, France..
    Joint Gaussian dictionary learning and tomographic reconstruction2022In: Inverse Problems, ISSN 0266-5611, E-ISSN 1361-6420, Vol. 38, no 10, article id 105010Article in journal (Refereed)
    Abstract [en]

    This paper studies ill-posed tomographic imaging problems where the image is sparsely represented by a non-negative linear combination of Gaussians. Our main contribution is to develop a scheme for directly recovering the Gaussian mixture representation of an image from tomographic data, which here is modeled as noisy samples of the parallel-beam ray transform. An important aspect of this non-convex reconstruction problem is the choice of initial guess. We propose an initialization procedure that is based on a filtered back projection type of operator tailored for the Gaussian dictionary. This operator can be evaluated efficiently using an approximation of the Riesz-potential of an anisotropic Gaussian which is based on an exact closed form expression for the Riesz-potential of an isotropic Gaussian. The proposed method is evaluated on simulated data.

12 1 - 50 of 59
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf