Learned image reconstruction has become a pillar in computational imaging and inverse problems. Among the most successful approaches are learned iterative networks, which are formulated by unrolling classical iterative optimization algorithms for solving variational problems. While the underlying algorithm is usually formulated in the functional analytic setting, learned approaches are often viewed as purely discrete. In this survey we present a unified operator view for learned iterative networks. Specifically, we formulate a learned reconstruction operator, defining how to compute , and separately the learning problem, which defines what to compute . In this setting we present common approaches and show that many approaches are closely related in their core. We review linear as well as non-linear inverse problems in this framework and present a short numerical study to conclude.

This article provides an overview of Jan Boman's illustrious seventy year career as an approximation theorist, microlocal analyst, and integral geometer. We will include his main mathematical themes and some personal observations.

Direct reconstruction of macromolecular structures from cryogenic electron microscopy (Cryo-EM) data has shown to be a challenge both in the homogeneous and heterogeneous setting. In this work we propose a new direct reconstruction method based on a combination of recent developments on protein geometry and orientation estimation. Even though this method is set up for the homogeneous setting, we aim to gain insight into challenges that atomistic methods have been facing for the heterogeneous case. In numerical experiments we observe that the method is able to recover the structure to almost inter-atomic resolution from as few as 100 2D Cryo-EM images due to the strong bias the regularizer gives. We conclude this work with a discussion on how the obtained results indicate possibilities and challenges for the generalization to the heterogeneous case.

We address recovery of the three-dimensional backbone structure of single polypeptide proteins from single-particle cryo--electron microscopy (Cryo-SPA) data. Cryo-SPA produces noisy tomographic projections of electrostatic potentials of macromolecules. From these projections, we use methods from shape analysis to recover the three-dimensional backbone structure. Thus, we view the reconstruction problem as an indirect matching problem, where a point cloud representation of the protein backbone is deformed to match two-dimensional tomography data. The deformations are obtained via the action of a matrix Lie group. By selecting a deformation energy, the optimality conditions are obtained, which lead to computational algorithms for optimal deformations. We showcase our approach on synthetic data, for which we recover the three-dimensional structure of the backbone.

Objective. Proton therapy treatment planning currently needs to account for relatively large range uncertainty margins primarily due to the semi-empirical calibration of the treatment planning x-ray computed tomography (CT) to proton stopping power relative to water (RSP). Proton radiography enables direct measurement of integral RSP, offering potential to improve calibration and reduce these uncertainties. Approach. Three deep neural network architectures are trained to infer a patient RSP map using the treatment planning CT and proton radiographies, assuming straight proton trajectories. The best suited architecture is then evaluated on more realistic, clinical-like data generated with Monte Carlo simulations. Three idealized proton imaging detectors are simulated: single particle tracking (SPT), a pixelated energy-resolving imager (PERI) and a proton-integrating detector (PID). Main results. The learned primal dual (LPD) architetcure performs best in the simplified imaging scenario. In the more realistic scenario, the initial calibration error (median absolute percentage error of 2.24% across the test set) is reduced using only two projections across all detector types. SPT and PERI reach similar performance (1.10%/1.12%), followed by PID (1.30%). Restricting the LPD to the calibration task by incorporating prior knowledge of the functional relationship of Hounsfield Units (HUs) and RSP further improves calibration performance. For SPT, conventional optimization on detector data acquired for individual protons outperformed the data-driven method (0.16% vs. 1.10%). However, for PERI and PID, the data-driven approach (1.12%/1.30%) slightly outperformed conventional optimization (1.63%/1.72%). Significance. To our knowledge, this is the first study to apply a deep learning-based approach fusing proton radiographies and treatment planning CT data for improved RSP calibration. The method achieves lower calibration errors than idealized, conventional calibration curve optimization on PERIs and PIDs-detector types that offer a promising path for clinical adoption due to their lower complexity and cost compared to SPT systems.

In the wood industry, logs are commonly quality screened by discrete X-ray scans on a moving conveyor belt from a few source positions. Typically, the measurements are obtained in a single two-dimensional (2D) plane (a "slice") by a sequential scanning geometry. The data from each slice alone does not carry sufficient information for a three-dimensional tomographic reconstruction in which biological features of interest in the log are well preserved. In the present work, we propose a learned iterative reconstruction method based on the Learned Primal-Dual neural network, suited for sequential scanning geometries. Our method accumulates information between neighbouring slices, instead of only accounting for single slices during reconstruction. Evaluations were performed by training U-Nets on segmentation of knots (branches), which are crucial features in wood processing. Our quantitative and qualitative evaluations show that with as few as five source positions our method yields reconstructions of logs that are sufficiently accurate to identify biological features like knots (branches), heartwood and sapwood.

In this study, we propose a deep learning based two-stage breast CT reconstruction in the image domain. Unlike most methods, we use two separate models to improve the Breast CT image quality. In the first stage, a deep learning-based denoiser was used to remove the noise. In the second stage, a deep learning based image enhancement model is used to improve the image quality. We evaluated the proposed method on the AAPM 2021 sparse view CT reconstruction challenge dataset.1 The experimental results demonstrate that the proposed method performs better than all comparison methods.

Deep learning based computed tomography (CT) reconstruction has demonstrated outstanding performance on simulated 2D low-dose CT data. This applies in particular to domain adapted neural networks, which incorporate a handcrafted physics model for CT imaging. Empirical evidence shows that employing such architectures reduces the demand for training data and improves upon generalization. However, their training requires large computational resources that quickly become prohibitive in 3D helical CT, which is the most common acquisition geometry used for medical imaging. This paper modifies a domain adapted neural network architecture, the Learned Primal-Dual (LPD), so that it can be trained and applied to reconstruction in this setting. The main challenge is to reduce the GPU memory requirements during the training, while keeping the computational time within practical limits. Furthermore, clinical data also comes with other challenges not accounted for in simulations, like errors in flux measurement, resolution mismatch and, most importantly, the absence of the real ground truth. To the best of our knowledge, this work is the first to apply an unrolled deep learning architecture for reconstruction on full-sized clinical data, like those in the Low dose CT image and projection data set (LDCT).

We propose several deep -learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward -backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step -size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.