kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Bathymetric Surveying with Imaging Sonar Using Neural Volume Rendering
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0001-8387-9951
(Monterey Bay Aquarium Research Institute, Moss Landing, CA, USA)
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0003-1189-6634
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0002-7796-1438
(English)Manuscript (preprint) (Other academic)
Abstract [en]

This research addresses the challenge of estimating bathymetry from imaging sonars where the state-of-the-art works have primarily relied on either supervised learning with ground-truth labels or surface rendering based on the Lambertian assumption.In this letter, we propose a novel, self-supervised framework based on volume rendering for reconstructing bathymetry using forward-looking sonar (FLS) data collected during standard surveys. We represent the seafloor as a neural heightmap encapsulated with a parametric multi-resolution hash encoding scheme and model the sonar measurements with a differentiable renderer using sonar volumetric rendering employed with hierarchical sampling techniques. Additionally, we model the horizontal and vertical beam patterns and estimate them jointly with the bathymetry. We evaluate the proposed method quantitatively on simulation and field data collected by remotely operated vehicles (ROVs) during low-altitude surveys. Results show that the proposed method outperforms the current state-of-the-art approaches that use imaging sonars for seabed mapping. We also demonstrate that the proposed approach can potentially be used to increase the resolution of a low-resolution prior map with FLS data from low-altitude surveys.

National Category
Robotics and automation
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-346441OAI: oai:DiVA.org:kth-346441DiVA, id: diva2:1857869
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Swedish Foundation for Strategic Research, IRC15-0046
Note

QC 20240520

Available from: 2024-05-14 Created: 2024-05-14 Last updated: 2025-02-09Bibliographically approved
In thesis
1. Bathymetric Surveying Through Neural Inverse Sonar Modeling
Open this publication in new window or tab >>Bathymetric Surveying Through Neural Inverse Sonar Modeling
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Autonomous underwater vehicles (AUVs) equipped with sidescan sonars (SSS) and remotely operated vehicles (ROVs) equipped with forward looking sonars (FLS)  have become vital tools for many underwater applications, among which, bathymetric mapping is one of the most important yet challenging tasks. Providing high-resolution imagery, SSS and FLS are particularly suitable for compact, low-cost and scalable vehicles, however their linear array design makes their measurements ambiguous in elevation. This limitation, makes inverse sonar modeling necessary for accurate and detailed bathymetric surveying in open sea missions. Solving such an inverse problem is usually ill-posed, even with repeated observations from different distances and viewpoints. Thus, this thesis has focused on learning-based approaches to sonar modeling, aiming to leverage advances in recent deep learning.

We present our contributions in three areas tackling this inverse sonar modeling. First, our work on learning SSS model with data-driven approaches shows how convolutional neural networks (CNNs) can be successfully applied to learn the inverse model in an end-to-end, supervised-learning fashion. We further show how to fuse different CNN estimates depending on the representations used for the seabed. We demonstrate that with an explicit grid representation, the uncertainty estimates of CNN's predictions can be of help in the fusion, while an implicit neural representation, specifically, a neural heightmap parameterized by multi-layer perceptrons (MLPs) could handle the fusion implicitly by posing the problem as a global optimization.Secondly, we leverage the methodology of representing the seabed with implicit neural representations and propose to use a Lambertian model based on surface rendering for sonar modeling. The proposed approach does not require collecting ground truth bathymetry and can be used on different datasets with different sensor setups. The experiments also demonstrate how the approach can converge  to a self-consistent map without any external bathymetric data.Finally, we adapt two differentiable volume rendering techniques  in  computer graphics to sonar modeling and show their advantages over surface rendering  on accurately modeling the physics behind the sonar ensonification process. In specific, a soft rasterization-based renderer with explicit mesh representations, and a ray-casting-based volumetric rendering with implicit neural representations with parametric encodings. The latter solution, in particular, is capable of leveraging deep learning advances without Lambertian approximation.  Our results show the proposed approach not only outperforms surface rendering solutions but its parametric encoding also allows it to outperform volume rendering methods with non-parametric encodings. We also demonstrate the potential application of increasing the resolution of  a low-resolution prior map with FLS data from low-altitude surveys.

Abstract [sv]

Autonomous underwater vehicles (AUVs) utrustade med sidescan sonars (SSS) och remotely operated behicles (ROVs) utrustade med forward looking sonars (FLS) har blivit avgörande verktyg för många undervattensapplikationer. Ett av dessa applikationer är batymetrisk kartläggning som är en av de viktigaste men samtidigt mest utmanande undervattensapplikationen. Genom att tillhandahålla högupplöst bildmateriel är SSS och FLS särskilt lämpliga verkty för kompakta, kostnadseffektiva och skalbara fordon. En begräsning i FLS samt SSS fasstyrda grupp ekolod design är att deras mätningar blir tvetydiga i höjdled.Denna begränsning gör invers sonarmodellering nödvändig för noggrann och detaljerad batymetrisk kartläggning i uppdrag på öppet hav.Att lösa ett sådant invers problem är vanligtvis en svår uppgift, även med upprepade observationer från olika avstånd och synvinklar. Därför har denna avhandling fokuserat på inlärnings baserade tillvägagångssätt för sonarmodellering i syfte att dra nytta av framsteg inom nyligen utvecklad djupinlärningsteknik.

Vi presenterar våra bidrag inom tre områden som hanterar invers sonarmodellering.Först visar vårt arbete med en inlärningsbaserad SSS-modell och datadrivna metoder hur convolutional neural networks (CNNs) framgångsrikt kan användas för att lära sig den inversa modellen på ett Väglett sätt.Vidare visar vi hur olika CNN-estimat kan kombineras beroende på vilken representation av havsbotten som används. Vi demonstrerar att med en explicit gitterrepresentation kan osäkerhetsestimat från CNN-förutsägelser vara till hjälp vid sammanslagningen. Med en implicit neural representation, specifikt en neural höjdkarta parametriserad av flerlagers perceptroner (MLPs), kan sammanslagningen hanteras implicit genom att formulera problemet som en global optimeringsfråga.För det andra utnyttjar vi metoden att representera havsbotten med implicita neurala representationer och föreslår användningen av en lambertiell modell baserad på ytritning för sonarmodellering. Detta tillvägagångssätt kräver inte insamling av verklig batymetri och kan användas på olika dataset med olika sensorsystem. Experimenten visar också att metoden kan konvergera till en självkonsistent karta utan några externa batymetriska data.Slutligen anpassar vi två olika typer av differentierbara volymrenderingstekniker inom datorgrafik till sonarmodellering och visar deras fördelar över ytritning för att noggrant modellera fysiken bakom sonarens insonifieringsprocess. Specifikt använder vi en mjuk rasteriseringsbaserad renderare med explicita nätrepresentationer och en strålspårningsbaserad volymrendering med implicita neurala representationer och parametriska kodningar. Den senare lösningen kan särskilt dra nytta av framsteg inom djupinlärning utan lambertiell approximation.Våra resultat visar att det föreslagna tillvägagångssättet inte bara överträffar lösningar baserade på ytritning, utan dess parametriska kodning gör det även möjligt att överträffa volymrenderingsmetoder med icke-parametriska kodningar. Vi demonstrerar också den potentiella tillämpningen av att öka upplösningen på en karta med låg upplösning med FLS-data från undersökningar på låg höjd.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. p. xv, 50
Series
TRITA-EECS-AVL ; 2024:38
National Category
Robotics and automation
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-346474 (URN)978-91-8040-912-4 (ISBN)
Public defence
2024-06-05, https://kth-se.zoom.us/j/62947919928, E3, Osquars backe 2, Stockholm, 14:00 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

QC 20240516

Available from: 2024-05-16 Created: 2024-05-15 Last updated: 2025-02-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Xie, YipingBore, NilsFolkesson, John

Search in DiVA

By author/editor
Xie, YipingBore, NilsFolkesson, John
By organisation
Robotics, Perception and Learning, RPL
Robotics and automation

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 73 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf