In this article, we propose a novel data-driven approach for high-resolution bathymetric reconstruction from sidescan. Sidescan sonar intensities as a function of range do contain some information about the slope of the seabed. However, that information must be inferred. In addition, the navigation system provides the estimated trajectory, and normally, the altitude along this trajectory is also available. From these, we obtain a very coarse seabed bathymetry as an input. This is then combined with the indirect but high-resolution seabed slope information from the sidescan to estimate the full bathymetry. This sparse depth could be acquired by single-beam echo sounder, Doppler velocity log, and other bottom tracking sensors or bottom tracking algorithm from sidescan itself. In our work, a fully convolutional network is used to estimate the depth contour and its aleatoric uncertainty from the sidescan images and sparse depth in an end-to-end fashion. The estimated depth is then used together with the range to calculate the point's three-dimensional location on the seafloor. A high-quality bathymetric map can be reconstructed after fusing the depth predictions and the corresponding confidence measures from the neural networks. We show the improvement of the bathymetric map gained by using sparse depths with sidescan over estimates with sidescan alone. We also show the benefit of confidence weighting when fusing multiple bathymetric estimates into a single map.

Sidescan sonar intensity encodes information about changes in the surface normal of the seabed. However, other factors such as seabed geometry as well as its material composition also affect the return intensity. One can model these intensity changes in a forward direction from the surface normals from a bathymetric map and physical properties to the measured intensity, or alternatively one can use an inverse model which starts from the intensities and models the surface normals. Here, we use an inverse model which leverages deep learning's ability to learn from data; a convolutional neural network is used to estimate the surface normal from the sidescan. Once this information is estimated, a bathymetric map can be reconstructed through an optimization framework that also includes altimeter readings to provide a sparse depth profile as a constraint. Implicit neural representation learning was recently proposed to represent the bathymetric map in such an optimization framework. In this article, we use a neural network to represent the map and optimize it under constraints of altimeter points and estimated surface normal from sidescan. By fusing multiple observations from different angles from several sidescan lines, the estimated results are improved through optimization. We demonstrate the efficiency and scalability of the approach by reconstructing a high-quality bathymetry using sidescan data from a large sidescan survey. We compare the proposed data-driven inverse model approach of modeling a sidescan with a forward Lambertian model. We assess the quality of each reconstruction by comparing it with data constructed from a multibeam sensor.

Sidescan sonar is a small and cost-effective sensing solution that can be easily mounted on most vessels. Historically, it has been used to produce high-definition acoustical images that experts may use to identify targets on the seafloor or in the water column. While solutions have been proposed to produce bathymetry solely from sidescan, or in conjunction with multibeam, they have had limited impact. This is partly a result of mostly being limited to single survey lines. In this article, we propose a modern, scalable solution to create high quality survey-scale bathymetry from many sidescan survey lines. By incorporating multiple observations of the same place, results can be improved as the estimates reinforce each other. Our method is based on sinusoidal representation networks, a recent advance in neural representation learning. We demonstrate the scalability of the approach by producing bathymetry from a large sidescan survey. The resulting quality is demonstrated by comparing to data collected with a high-precision multibeam sensor.

This paper outlines challenges and opportunities in operating underwater robots (so-called AUVs) on a seaweed farm. The need is driven by an emerging aquaculture industry on the Swedish west coast where large-scale seaweed farms are being developed. In this paper, the operational challenges are described and key technologies in using autonomous systems as a core part of the operation are developed and demonstrated. The paper presents a system and methods for operating an AUV in the seaweed farm, including initial localization of the farm based on a prior estimate and dead-reckoning navigation, and the subsequent scanning of the entire farm. Critical data from sidescan sonars for algorithm development are collected from real environments at a test site in the ocean, and the results are demonstrated in a simulated seaweed farm setup.

Sidescan sonar is a small and low-cost sensor that can be mounted on most unmanned underwater vehicles (UUVs) and unmanned surface vehicles (USVs). It has the advantages of high resolution and wide coverage, which could be valuable in providing an efficient and cost-effective solution for obtaining the bathymetry when bathymetric data are unavailable. This work proposes a method of reconstructing bathymetry using only sidescan data from large-scale surveys by formulating the problem as a global optimization, where a Sinusoidal Representation Network (SIREN) is used to represent the bathymetry and the albedo and the beam profile are jointly estimated based on a Lambertian scattering model. The assessment of the proposed method is conducted by comparing the reconstructed bathymetry with the bathymetric data collected with a high-resolution multi-beam echo sounder (MBES). An error of 20 cm on the bathymetry is achieved from a large-scale survey. The proposed method proved to be an effective way to reconstruct bathymetry from sidescan sonar data when high-accuracy positioning is available. This could be of great use for applications such as surface vehicles with Global Navigation Satellite System (GNSS) to obtain high-quality bathymetry in shallow water or small autonomous underwater vehicles (AUVs) if simultaneous localization and mapping (SLAM) can be applied to correct the navigation estimate.

Recent advances in differentiable rendering, which allow calculating the gradients of 2D pixel values with respect to 3D object models, can be applied to estimation of the model parameters by gradient-based optimization with only 2D supervision. It is easy to incorporate deep neural networks into such an optimization pipeline, allowing the leveraging of deep learning techniques. This also largely reduces the requirement for collecting and annotating 3D data, which is very difficult for applications, for example when constructing geometry from 2D sensors. In this work, we propose a differentiable renderer for sidescan sonar imagery. We further demonstrate its ability to solve the inverse problem of directly reconstructing a 3D seafloor mesh from only 2D sidescan sonar data.

In this study, we considered the problem of underwater image classification in the context of underwater pipeline inspection. This task is particularly more difficult than many classification tasks due to the subtle differences between classes, the inherent imbalance in the occurrence of the classes and the variability caused by water clarity. We experimented with both transformer and CNN architectures while for the latter, we also employed auxiliary learning as well as capitalizing on the multiview aspect on the underwater pipeline capturing setup. Additionally, we also adopted a DNN interpretability approach to locate the regions relevant to the predictions using solely image-level annotations. Finally, the extracted explainability cues were integrated into the training process with the purpose of producing more robust predictions and more complete explanations.

Sidescan sonar has been used for maritime surveys since the mid-20th century. Due to its wide swath coverage and the sharpness of the produced images, it is an invaluable tool still to this day. When simulating sidescan data, there is a tradeoff between the quality of the produced images, the fidelity of the environment simulation, and the complexity of the sidescan model. In this article, we propose data-driven models as a way of removing some of this tradeoff. Using recently proposed conditional generative adversarial nets, we create a generative model that takes the environment as input, and produces realistic sidescan measurements. We show that end-to-end learning of flexible models allows simulating more complex sidescan data than would otherwise be possible given only geometric bathymetry.

Cyber-physical systems (CPSs) comprise a network of sensors and actuators that are integrated with a computing and communication core. Hydrobatic Autonomous Underwater Vehicles (AUVs) can be efficient and agile, offering new use cases in ocean production, environmental sensing and security. In this paper, a CPS concept for hydrobatic AUVs is validated in real-world field trials with the hydrobatic AUV SAM developed at the Swedish Maritime Robotics Center (SMaRC). We present system integration of hardware systems, software subsystems for mission planning using Neptus, mission execution using behavior trees, flight and trim control, navigation and dead reckoning. Together with the software systems, we show simulation environments in Simulink and Stonefish for virtual validation of the entire CPS. Extensive field validation of the different components of the CPS has been performed. Results of a field demonstration scenario involving the search and inspection of a submerged Mini Cooper using payload cameras on SAM in the Baltic Sea are presented. The full system including the mission planning interface, behavior tree, controllers, dead-reckoning and object detection algorithm is validated. The submerged target is successfully detected both in simulation and reality, and simulation tools show tight integration with target hardware.

Sidescan sonar images are 2D representations of the seabed. The pixel location encodes distance from the sonar and along track coordinate. Thus one dimension is lacking for generating bathymetric maps from sidescan. The intensities of the return signals do, however, contain some information about this missing dimension. Just as shading gives clues to depth in camera images, these intensities can be used to estimate bathymetric profiles. The authors investigate the feasibility of using data driven methods to do this estimation. They include quantitative evaluations of two pixel-to-pixel convolutional neural networks trained as standard regression networks and using conditional generative adversarial network loss functions. Some interesting conclusions are presented as to when to use each training method.