kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Temporal Modeling of Dynamic 3D Scenes for Self-supervised Scene Flow
KTH, School of Electrical Engineering and Computer Science (EECS), Robotics, Perception and Learning.ORCID iD: 0000-0002-7882-948X
2026 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The physical world is never still. This thesis studies how the 3D motion of a dynamic scene can be estimated from point clouds without human annotation. Scene flow assigns a displacement vector to every observed point, but dense motion labels are prohibitively expensive, which has kept the representation from practical use.

We approach the problem through the physical nature of motion. Unlike the categories behind detection or segmentation, motion is not defined by people; it is a physical quantity of the scene, and three of its properties are exploited here. It is measurable from geometry, so supervision can be derived from observations rather than annotators. It is continuous in time, so past frames constrain the present one. And it depends on kinematics rather than appearance, so it can be generated at scale in simulation and transferred to real sensors.

We first show that feed-forward models trained with point-wise matching systematically underestimate dynamic motion, because nearest-neighbor correspondences on large moving objects point to the wrong surface. Separating static structure from dynamic objects and enforcing object-level kinematic constraints yields reliable label-free supervision from a pair of frames. To exploit the temporal continuity of motion, we then extend supervision to multiple frames, which raises two problems: computation that grows with the temporal window, which we keep constant through a compact residual representation, and correspondences that shift abruptly across frames, which we resolve by aggregating the most temporally consistent cues from a candidate pool, keeping multi-frame self-supervision stable. Even with such carefully designed objectives, self-supervision still relies on proxy signals derived from geometric assumptions. These signals remain noisy under sensor sparsity and occlusion, so scaling unlabeled real data yields diminishing returns. Because motion depends on kinematics rather than appearance, we instead turn to simulation, prioritizing motion complexity over sensor realism to generate large-scale synthetic data with noise-free labels derived directly from simulator states. The resulting motion prior generalizes zero-shot across sensors and sharply reduces the need for real annotations. Finally, we show that reliable flow estimates act as point-level velocity fields that remove motion distortion from raw LiDAR sweeps.

Together, these results show that the physical objectivity of motion is more than a conceptual idea: it is a property that can be exploited to make dynamic scene understanding scalable and generalizable without human supervision.

Abstract [sv]

Den fysiska världen står aldrig stilla. Denna avhandling studerar hur tredimensionell rörelse i en dynamisk scen kan skattas från punktmoln utan mänsklig annotering. Scenflöde tilldelar varje observerad punkt en förskjutningsvektor. Att annotera rörelsen för varje punkt i stora mängder punktmolnsdata är orimligt kostsamt. Detta har hindrat scenflöde från att komma till praktisk användning.

Vi angriper problemet utifrån rörelsens fysikaliska natur. Till skillnad från de mänskligt definierade kategorierna som används för detektion och segmentering är scenflöde en fysikalisk storhet i scenen. Vi utnyttjar tre av dess egenskaper. Rörelse är mätbar utifrån geometri, så handledning kan härledas ur observationerna i stället för från annoteringar. Rörelse är kontinuerlig i tiden, så tidigare rörelse begränsar den aktuella. Rörelse beror dessutom på kinematik snarare än utseende, så data kan genereras i stor skala genom simulering och överföras till verkliga sensorer.

Vi visar först att framåtmatande modeller som tränas med punktvis matchning systematiskt underskattar dynamisk rörelse, eftersom närmaste-granne-korrespondenser på stora rörliga objekt pekar mot fel yta. Genom att separera statisk struktur från dynamiska objekt och införa kinematiska bivillkor på objektnivå erhålls tillförlitlig annoteringsfri handledning för två på varandra följande bildrutor. För att utnyttja att rörelse är kontinuerlig i tid utvidgar vi sedan handledningen till flera bildrutor, vilket skapar två utmaningar: beräkningskostnaden växer med tidsfönstrets storlek, vilket vi håller konstant genom en kompakt residualrepresentation, och korrespondenserna växlar abrupt mellan bildrutor, vilket vi löser genom att ur en kandidatmängd aggregera de ledtrådar som är mest konsistenta över tid, så att självhandledning över flera bildrutor förblir stabil. Även med sådana noggrant utformade målfunktioner bygger självhandledning fortfarande på indirekta träningssignaler som härleds ur geometriska antaganden. Dessa signaler förblir brusiga vid glesa sensormätningar och ocklusion, och att skala upp mängden oannoterade verkliga data ger därför allt mindre förbättring. Eftersom rörelse beror på kinematik snarare än utseende vänder vi oss i stället till simulering. Vi prioriterar rörelsekomplexitet framför sensorrealism och genererar storskaliga syntetiska data med brusfria etiketter som härleds direkt ur simulatorns tillstånd. Den resulterande förhandsmodellen för rörelse generaliserar mellan olika sensorer utan ytterligare träning (zero-shot) och minskar kraftigt behovet av annoterade verkliga data. Slutligen visar vi att tillförlitliga flödesskattningar fungerar som hastighetsfält på punktnivå och kan korrigera rörelsedistorsion i råa LiDAR-svep.

Sammantaget visar resultaten att rörelsens objektiva, fysikaliska natur inte bara är en konceptuellt tilltalande idé utan också kan utnyttjas för att göra dynamisk scenförståelse skalbar och generaliserbar utan mänsklig handledning.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2026. , p. ix, 38
Series
TRITA-EECS-AVL ; 2026:82
National Category
Computer Vision and Learning Systems
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-388958ISBN: 978-91-8106-709-5 (print)OAI: oai:DiVA.org:kth-388958DiVA, id: diva2:2105419
Public defence
2026-10-23, https://kth-se.zoom.us/j/68797563146, Kollegiesalen, Brinellvägen 6, Stockholm, 09:00 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP), 30014J
Note

QC 20260928

Available from: 2026-09-28 Created: 2026-09-25 Last updated: 2026-09-28Bibliographically approved
List of papers
1. DUFOMap: Efficient Dynamic Awareness Mapping
Open this publication in new window or tab >>DUFOMap: Efficient Dynamic Awareness Mapping
2024 (English)In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 9, no 6, p. 5038-5045Article in journal (Refereed) Published
Abstract [en]

The dynamic nature of the real world is one of the main challenges in robotics. The first step in dealing with it is to detect which parts of the world are dynamic. A typical benchmark task is to create a map that contains only the static part of the world to support, for example, localization and planning. Current solutions are often applied in post-processing, where parameter tuning allows the user to adjust the setting for a specific dataset. In this letter, we propose DUFOMap, a novel dynamic awareness mapping framework designed for efficient online processing. Despite having the same parameter settings for all scenarios, it performs better or is on par with state-of-the-art methods. Ray casting is utilized to identify and classify fully observed empty regions. Since these regions have been observed empty, it follows that anything inside them at another time must be dynamic. Evaluation is carried out in various scenarios, including outdoor environments in KITTI and Argoverse 2, open areas on the KTH campus, and with different sensor types. DUFOMap outperforms the state of the art in terms of accuracy and computational efficiency.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Mapping, object detection, robotics and automation in construction, segmentation and categorization
National Category
Robotics and automation Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-366806 (URN)10.1109/LRA.2024.3387658 (DOI)001205782500001 ()2-s2.0-85190348409 (Scopus ID)
Note

QC 20250710

Available from: 2025-07-10 Created: 2025-07-10 Last updated: 2026-09-25Bibliographically approved
2. DeFlow: Decoder of Scene Flow Network in Autonomous Driving
Open this publication in new window or tab >>DeFlow: Decoder of Scene Flow Network in Autonomous Driving
Show others...
2024 (English)In: 2024 IEEE International Conference on Robotics and Automation, ICRA 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 2105-2111Conference paper, Published paper (Refereed)
Abstract [en]

Scene flow estimation determines a scene's 3D motion field, by predicting the motion of points in the scene, especially for aiding tasks in autonomous driving. Many networks with large-scale point clouds as input use voxelization to create a pseudo-image for real-time running. However, the voxelization process often results in the loss of point-specific features. This gives rise to a challenge in recovering those features for scene flow tasks. Our paper introduces DeFlow which enables a transition from voxel-based features to point features using Gated Recurrent Unit (GRU) refinement. To further enhance scene flow estimation performance, we formulate a novel loss function that accounts for the data imbalance between static and dynamic points. Evaluations on the Argoverse 2 scene flow task reveal that DeFlow achieves state-of-the-art results on large-scale point cloud data, demonstrating that our network has better performance and efficiency compared to others. The code is available at https://github.com/KTH-RPL/deflow.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
National Category
Computer graphics and computer vision Signal Processing
Identifiers
urn:nbn:se:kth:diva-367438 (URN)10.1109/ICRA57147.2024.10610278 (DOI)001294576201114 ()2-s2.0-85188850619 (Scopus ID)
Conference
2024 IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13-17, 2024
Note

Part of ISBN 9798350384574

QC 20250718

Available from: 2025-07-18 Created: 2025-07-18 Last updated: 2026-09-25Bibliographically approved
3. SeFlow: A Self-supervised Scene Flow Method in Autonomous Driving
Open this publication in new window or tab >>SeFlow: A Self-supervised Scene Flow Method in Autonomous Driving
Show others...
2025 (English)In: COMPUTER VISION-ECCV 2024, PT I / [ed] Roth, S Russakovsky, O Sattler, T Varol, G Leonardis, A Ricci, E, Springer Nature , 2025, Vol. 15059, p. 353-369Conference paper, Published paper (Refereed)
Abstract [en]

Scene flow estimation predicts the 3D motion at each point in successive LiDAR scans. This detailed, point-level, information can help autonomous vehicles to accurately predict and understand dynamic changes in their surroundings. Current state-of-the-art methods require annotated data to train scene flow networks and the expense of labeling inherently limits their scalability. Self-supervised approaches can overcome the above limitations, yet face two principal challenges that hinder optimal performance: point distribution imbalance and disregard for object-level motion constraints. In this paper, we propose SeFlow, a self-supervised method that integrates efficient dynamic classification into a learning-based scene flow pipeline. We demonstrate that classifying static and dynamic points helps design targeted objective functions for different motion patterns. We also emphasize the importance of internal cluster consistency and correct object point association to refine the scene flow estimation, in particular on object details. Our real-time capable method achieves state-of-the-art performance on the self-supervised scene flow task on Argoverse 2 and Waymo datasets. The code is open-sourced at https://github.com/KTH-RPL/SeFlow.

Place, publisher, year, edition, pages
Springer Nature, 2025
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 15059
Keywords
3D scene flow, self-supervised, autonomous driving, large-scale point cloud
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-357529 (URN)10.1007/978-3-031-73232-4_20 (DOI)001346378300020 ()2-s2.0-85206389477 (Scopus ID)
Conference
18th European Conference on Computer Vision (ECCV), SEP 29-OCT 04, 2024, Milan, ITALY
Note

Part of ISBN 978-3-031-73231-7; 978-3-031-73232-4

QC 20241209

Available from: 2024-12-09 Created: 2024-12-09 Last updated: 2026-09-25Bibliographically approved
4. DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method
Open this publication in new window or tab >>DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method
Show others...
2025 (English)In: Advances in Neural Information Processing Systems 38 (NeurIPS 2025) / [ed] Belgrave, D; Zhang, C; Lin, H; Pascanu, R; Koniusz, P; Ghassemi, M; Chen, N, Curran Associates, Inc. , 2025, Vol. 38, p. 88951-88975Conference paper, Published paper (Refereed)
Abstract [en]

Previous dominant methods for scene flow estimation focus mainly on input from two consecutive frames, neglecting valuable information in the temporal domain. While recent trends shift towards multi-frame reasoning, they suffer from rapidly escalating computational costs as the number of frames grows. To leverage temporal information more efficiently, we propose DeltaFlow (ΔFlow), a lightweight 3D framework that captures motion cues via a Δ scheme, extracting temporal features with minimal computational cost, regardless of the number of frames. Additionally, scene flow estimation faces challenges such as imbalanced object class distributions and motion inconsistency. To tackle these issues, we introduce a Category-Balanced Loss to enhance learning across underrepresented classes and an Instance Consistency Loss to enforce coherent object motion, improving flow accuracy. Extensive evaluations on the Argoverse 2, Waymo and nuScenes datasets show that ΔFlow achieves state-of-the-art performance with up to 22% lower error and 2× faster inference compared to the next-best multi-frame supervised method, while also demonstrating a strong cross-domain generalization ability. The code is open-sourced at https://github.com/Kin-Zhang/DeltaFlow along with trained model weights.

Place, publisher, year, edition, pages
Curran Associates, Inc., 2025
National Category
Computer graphics and computer vision
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-384233 (URN)10.52202/085713-2975 (DOI)
Conference
39th Conference on Neural Information Processing Systems (NeurIPS 2025), San Diego, California, USA, 2-7 Dec 2025, and Mexico City, Mexico, 30 Nov - 5 Dec 2025
Note

QC 20260928

Available from: 2026-09-25 Created: 2026-09-25 Last updated: 2026-09-28Bibliographically approved
5. TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation
Open this publication in new window or tab >>TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation
Show others...
2026 (English)In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, p. 3667-3676Conference paper, Published paper (Refereed)
Abstract [en]

Self-supervised feed-forward methods for scene flow estimation offer real-time efficiency, but their supervision from two-frame point correspondences is unreliable and often breaks down under occlusions. Multi-frame supervision has the potential to provide more stable guidance by incorporating motion cues from past frames, yet naive extensions of two-frame objectives are ineffective because point correspondences vary abruptly across frames, producing inconsistent signals. In the paper, we present TeFlow, enabling multi-frame supervision for feed-forward models by mining temporally consistent supervision. TeFlow introduces a temporal ensembling strategy that forms reliable supervisory signals by aggregating the most temporally consistent motion cues from a candidate pool built across multiple frames. Extensive evaluations demonstrate that TeFlow establishes a new state-of-the-art for self-supervised feed-forward methods, achieving performance gains of up to 33% on the challenging Argoverse 2 and nuScenes datasets. Our method performs on par with leading optimization-based methods, yet speeds up 150 times. The code is open-sourced at https://github.com/Kin-Zhang/TeFlow along with trained model weights.

National Category
Computer graphics and computer vision
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-384234 (URN)
Conference
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026), Colorado Convention Center, USA, 3-7, 2026
Note

QC 20260928

Available from: 2026-09-25 Created: 2026-09-25 Last updated: 2026-09-28Bibliographically approved
6. SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data
Open this publication in new window or tab >>SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data
2026 (English)In: Computer Vision – ECCV 2026 / [ed] Favaro, P; Kukelova, Z; Maki, A; Rohrbach, A; Schindler, K; Tombari, F, Cham: Springer Nature , 2026, Vol. 17016, p. 151-168Conference paper, Published paper (Refereed)
Abstract [en]

Reliable 3D dynamic perception requires models that can anticipate motion beyond predefined categories, yet progress is hindered by the scarcity of dense, high-quality motion annotations. While self-supervision on unlabeled real data offers a path forward, empirical evidence suggests that scaling unlabeled data fails to close the performance gap due to noisy proxy signals. In this paper, we propose a shift in paradigm: learning robust real-world motion priors entirely from scalable simulation. We introduce SynFlow, a data generation pipeline that generates large-scale synthetic dataset specifically designed for LiDAR scene flow. Unlike prior works that prioritize sensor-specific realism, SynFlow employs a motion-oriented strategy to synthesize diverse kinematic patterns across 4,000 sequences (∼940k frames), termed SynFlow-4k. This represents a 34× scale-up in annotated volume over existing real-world benchmarks. Our experiments demonstrate that SynFlow-4k provides a highly domain-invariant motion prior. In a zero-shot regime, models trained exclusively on our synthetic data generalize across multiple real-world benchmarks, rivaling in-domain supervised baselines on nuScenes and outperforming state-of-the-art methods on TruckScenes by 31.8%. Furthermore, SynFlow-4k serves as a label-efficient foundation: fine-tuning with only 5% of real-world labels surpasses models trained from scratch on the full available budget. We open-source the pipeline and dataset to facilitate research in generalizable 3D motion estimation. More detail can be found at https://kin-zhang.github.io/SynFlow.

Place, publisher, year, edition, pages
Cham: Springer Nature, 2026
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 17016
Keywords
Scene flow estimation, synthetic data, autonomous driving, LiDAR
National Category
Computer graphics and computer vision
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-384235 (URN)10.1007/978-3-032-37383-0_9 (DOI)2-s2.0-105052542916 (Scopus ID)
Conference
The 19th European Conference on Computer Vision ECCV 2026, Malmö, Sweden, 8-12 2026
Note

Part of ISBN 978-3-032-37383-0

QC 20260928

Available from: 2026-09-25 Created: 2026-09-25 Last updated: 2026-10-14Bibliographically approved
7. HiMo: High-Speed Objects Motion Compensation in Point Clouds
Open this publication in new window or tab >>HiMo: High-Speed Objects Motion Compensation in Point Clouds
Show others...
2025 (English)In: IEEE Transactions on robotics, ISSN 1552-3098, E-ISSN 1941-0468, Vol. 41, p. 5896-5911Article in journal (Refereed) Published
Abstract [en]

LiDAR point cloud is essential for autonomous vehicles, but motion distortions from dynamic objects degrade the data quality. While previous work has considered distortions caused by ego motion, distortions caused by other moving objects remain largely overlooked, leading to errors in object shape and position. This distortion is particularly pronounced in high-speed environments such as highways and in multi-LiDAR configurations, a common setup for heavy vehicles. To address this challenge, we introduce HiMo, a pipeline that repurposes scene flow estimation for non-ego motion compensation, correcting the representation of dynamic objects in point clouds. During the development of HiMo, we observed that existing self-supervised scene flow estimators often produce degenerate or inconsistent estimates under high-speed distortion. We further propose SeFlow++, a real-time scene flow estimator that achieves state-of-the-art performance on both scene flow and motion compensation. Since well-established motion distortion metrics are absent in the literature, we introduce two evaluation metrics: compensation accuracy at a point level and shape similarity of objects. We validate HiMo through extensive experiments on Argoverse 2, ZOD and a newly collected real-world dataset featuring highway driving and multi-LiDAR-equipped heavy vehicles. Our findings show that HiMo improves the geometric consistency and visual fidelity of dynamic objects in LiDAR point clouds, benefiting downstream tasks such as semantic segmentation and 3D detection. See https://kin-zhang.github.io/HiMo for more details.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
Keywords
Autonomous Driving Navigation, Computer Vision for Transportation, Motion Compensation, Range Sensing
National Category
Computer graphics and computer vision Robotics and automation Vehicle and Aerospace Engineering Signal Processing
Identifiers
urn:nbn:se:kth:diva-372474 (URN)10.1109/TRO.2025.3619042 (DOI)001600720400002 ()2-s2.0-105019222489 (Scopus ID)
Note

QC 20251107

Available from: 2025-11-07 Created: 2025-11-07 Last updated: 2026-09-25Bibliographically approved

Open Access in DiVA

kappa-web(4758 kB)197 downloads
File information
File name FULLTEXT01.pdfFile size 4758 kBChecksum SHA-512
9d3612553fcfc219ebc5b21e67768ee0149074b8859c287fb8f621fb95b0ddda68ee80ea7d84c87f3ba8596de47de6e6bcf7ab8b8f5706eaef1b52cdbdb60974
Type fulltextMimetype application/pdf

Authority records

Zhang, Qingwen

Search in DiVA

By author/editor
Zhang, Qingwen
By organisation
Robotics, Perception and Learning
Computer Vision and Learning Systems

Search outside of DiVA

GoogleGoogle Scholar
Total: 197 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 2130 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf