kth.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 63) Show all publications
Dong, Y., Cheng, X. & Pokorny, F. T. (2024). Characterizing Manipulation Robustness Through Energy Margin and Caging Analysis. IEEE Robotics and Automation Letters, 9(9), 7525-7532
Open this publication in new window or tab >>Characterizing Manipulation Robustness Through Energy Margin and Caging Analysis
2024 (English)In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 9, no 9, p. 7525-7532Article in journal (Refereed) Published
Abstract [en]

To develop robust manipulation policies, quantifying robustness is essential. Evaluating robustness in general manipulation, nonetheless, poses significant challenges due to complex hybrid dynamics, combinatorial explosion of possible contact interactions, global geometry, etc. This paper introduces an approach for evaluating manipulation robustness through energy margins and caging-based analysis. Our method assesses manipulation robustness by measuring the energy margin to failure and extends traditional caging concepts for dynamic manipulation. This global analysis is facilitated by a kinodynamic planning framework that naturally integrates global geometry, contact changes, and robot compliance. We validate the effectiveness of our approach in simulation and real-world experiments of multiple dynamic manipulation scenarios, highlighting its potential to predict manipulation success and robustness.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Robustness, End effectors, Task analysis, Robots, Measurement, Manipulator dynamics, Friction, Dexterous manipulation, in-hand manipulation, contact modeling, manipulation planning
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-351412 (URN)10.1109/LRA.2024.3418309 (DOI)001273087700014 ()2-s2.0-85197045618 (Scopus ID)
Note

QC 20240812

Available from: 2024-08-12 Created: 2024-08-12 Last updated: 2025-02-09Bibliographically approved
Zahid, M. & Pokorny, F. T. (2024). CloudGripper: An Open Source Cloud Robotics Testbed for Robotic Manipulation Research, Benchmarking and Data Collection at Scale. In: 2024 IEEE International Conference on Robotics and Automation, ICRA 2024: . Paper presented at 2024 IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13 2024 - May 17 2024 (pp. 12076-12082). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>CloudGripper: An Open Source Cloud Robotics Testbed for Robotic Manipulation Research, Benchmarking and Data Collection at Scale
2024 (English)In: 2024 IEEE International Conference on Robotics and Automation, ICRA 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 12076-12082Conference paper, Published paper (Refereed)
Abstract [en]

We present CloudGripper, an open source cloud robotics testbed, consisting of a scalable, space and cost-efficient design constructed as a rack of 32 small robot arm work cells. Each robot work cell is fully enclosed and features individual lighting, a low-cost Cartesian robot arm with an attached rotatable parallel jaw gripper and a dual camera setup for experimentation. The system design is focused on continuous operation and features a 10 Gbit/s network connectivity allowing for high throughput remote-controlled experimentation and data collection for robotic manipulation. Furthermore, CloudGripper is intended to form a community testbed to study the challenges of large scale machine learning and cloud and edge-computing in the context of robotic manipulation. In this work, we describe the mechanical design of the system, its initial software stack and evaluate the repeatability of motions executed by the proposed robot arm design. A local network API throughput and latency analysis is also provided. CloudGripper-Rope-100, a dataset of more than a hundred hours of randomized rope pushing interactions and approximately 4 million camera images is collected and serves as a proof of concept demonstrating data collection capabilities. A project website with more information is available at https://cloudgripper.org.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-353559 (URN)10.1109/ICRA57147.2024.10611548 (DOI)001369728002083 ()2-s2.0-85202441651 (Scopus ID)
Conference
2024 IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13 2024 - May 17 2024
Note

Part of ISBN [9798350384574]

QC 20240923

Available from: 2024-09-19 Created: 2024-09-19 Last updated: 2025-03-10Bibliographically approved
Jin, S., Wang, R., Zahid, M. & Pokorny, F. T. (2024). How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushing. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024: . Paper presented at 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024, Abu Dhabi, United Arab Emirates, October 14-18, 2024 (pp. 7391-7398). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushing
2024 (English)In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 7391-7398Conference paper, Published paper (Refereed)
Abstract [en]

As model and dataset sizes continue to scale in robot learning, the need to understand how the composition and properties of a dataset affect model performance becomes increasingly urgent to ensure cost-effective data collection and model performance. In this work, we empirically investigate how physics attributes (color, friction coefficient, shape) and scene background characteristics, such as the complexity and dynamics of interactions with background objects, influence the performance of Video Transformers in predicting planar pushing trajectories. We investigate three primary questions: How do physics attributes and background scene characteristics influence model performance? What kind of changes in attributes are most detrimental to model generalization? What proportion of fine-tuning data is required to adapt models to novel scenarios? To facilitate this research, we present CloudGripper-Push-1K, a large real-world vision-based robot pushing dataset comprising 1278 hours and 460,000 videos of planar pushing interactions with objects with different physics and background attributes. We also propose Video Occlusion Transformer (VOT), a generic modular video-transformer-based trajectory prediction framework which features 3 choices of 2D-spatial encoders as the subject of our case study. The dataset and source code are available at https://cloudgripper.org.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
National Category
Computer graphics and computer vision Robotics and automation
Identifiers
urn:nbn:se:kth:diva-359877 (URN)10.1109/IROS58592.2024.10802583 (DOI)001411890000704 ()2-s2.0-85216457245 (Scopus ID)
Conference
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024, Abu Dhabi, United Arab Emirates, October 14-18, 2024
Note

Part of ISBN 979-8-3503-7770-5

QC 20250213

Available from: 2025-02-12 Created: 2025-02-12 Last updated: 2025-04-25Bibliographically approved
Marchetti, G. L., Polianskii, V., Varava, A., Pokorny, F. T. & Kragic, D. (2023). An Efficient and Continuous Voronoi Density Estimator. In: Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023: . Paper presented at 26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023, Valencia, Spain, Apr 25 2023 - Apr 27 2023 (pp. 4732-4744). ML Research Press
Open this publication in new window or tab >>An Efficient and Continuous Voronoi Density Estimator
Show others...
2023 (English)In: Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023, ML Research Press , 2023, p. 4732-4744Conference paper, Published paper (Refereed)
Abstract [en]

We introduce a non-parametric density estimator deemed Radial Voronoi Density Estimator (RVDE). RVDE is grounded in the geometry of Voronoi tessellations and as such benefits from local geometric adaptiveness and broad convergence properties. Due to its radial definition RVDE is continuous and computable in linear time with respect to the dataset size. This amends for the main shortcomings of previously studied VDEs, which are highly discontinuous and computationally expensive. We provide a theoretical study of the modes of RVDE as well as an empirical investigation of its performance on high-dimensional data. Results show that RVDE outperforms other non-parametric density estimators, including recently introduced VDEs.

Place, publisher, year, edition, pages
ML Research Press, 2023
Series
Proceedings of Machine Learning Research, ISSN 2640-3498, ; 206
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-334436 (URN)001222727704044 ()2-s2.0-85165187458 (Scopus ID)
Conference
26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023, Valencia, Spain, Apr 25 2023 - Apr 27 2023
Note

QC 20241204

Available from: 2023-08-21 Created: 2023-08-21 Last updated: 2025-02-07Bibliographically approved
Gieselmann, R. & Pokorny, F. T. (2023). Expansive Latent Planning for Sparse Reward Offline Reinforcement Learning. In: Proceedings of The 7th Conference on Robot Learning: . Paper presented at The 7th Conference on Robot Learning, Atlanta, GA, Nov 6-9, 2023. Proceedings of Machine Learning Research
Open this publication in new window or tab >>Expansive Latent Planning for Sparse Reward Offline Reinforcement Learning
2023 (English)In: Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research , 2023Conference paper, Published paper (Refereed)
Abstract [en]

Sampling-based motion planning algorithms excel at searching global solution paths in geometrically complex settings. However, classical approaches, such as RRT, are difficult to scale beyond low-dimensional search spaces and rely on privileged knowledge e.g. about collision detection and underlying state distances. In this work, we take a step towards the integration of sampling-based planning into the reinforcement learning framework to solve sparse-reward control tasks from high-dimensional inputs. Our method, called VELAP, determines sequences of waypoints through sampling-based exploration in a learned state embedding. Unlike other sampling-based techniques, we iteratively expand a tree-based memory of visited latent areas, which is leveraged to explore a larger portion of the latent space for a given number of search iterations. We demonstrate state-of-the-art results in learning control from offline data in the context of vision-based manipulation under sparse reward feedback. Our method extends the set of available planning tools in model-based reinforcement learning by adding a latent planner that searches globally for feasible paths instead of being bound to a fixed prediction horizon. 

Place, publisher, year, edition, pages
Proceedings of Machine Learning Research, 2023
National Category
Computer graphics and computer vision Robotics and automation
Identifiers
urn:nbn:se:kth:diva-341581 (URN)001221201500001 ()2-s2.0-85184350420 (Scopus ID)
Conference
The 7th Conference on Robot Learning, Atlanta, GA, Nov 6-9, 2023
Note

QC 20231227

Available from: 2023-12-22 Created: 2023-12-22 Last updated: 2025-02-05Bibliographically approved
Kravchenko, A., Marchetti, G. L., Polianskii, V., Varava, A., Pokorny, F. T. & Kragic, D. (2022). Active Nearest Neighbor Regression Through Delaunay Refinement. In: Proceedings of the 39th International Conference on Machine Learning: . Paper presented at 39th International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 17-23 July, 2022 (pp. 11650-11664). MLResearch Press, 162
Open this publication in new window or tab >>Active Nearest Neighbor Regression Through Delaunay Refinement
Show others...
2022 (English)In: Proceedings of the 39th International Conference on Machine Learning, MLResearch Press , 2022, Vol. 162, p. 11650-11664Conference paper, Published paper (Refereed)
Abstract [en]

We introduce an algorithm for active function approximation based on nearest neighbor regression. Our Active Nearest Neighbor Regressor (ANNR) relies on the Voronoi-Delaunay framework from computational geometry to subdivide the space into cells with constant estimated function value and select novel query points in a way that takes the geometry of the function graph into account. We consider the recent state-of-the-art active function approximator called DEFER, which is based on incremental rectangular partitioning of the space, as the main baseline. The ANNR addresses a number of limitations that arise from the space subdivision strategy used in DEFER. We provide a computationally efficient implementation of our method, as well as theoretical halting guarantees. Empirical results show that ANNR outperforms the baseline for both closed-form functions and real-world examples, such as gravitational wave parameter inference and exploration of the latent space of a generative model.

Place, publisher, year, edition, pages
MLResearch Press, 2022
Series
Proceedings of Machine Learning Research, ISSN 2640-3498 ; 162
National Category
Computer Sciences Control Engineering
Identifiers
urn:nbn:se:kth:diva-319194 (URN)000900064901033 ()2-s2.0-85163127180 (Scopus ID)
Conference
39th International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 17-23 July, 2022
Note

QC 20230509

Available from: 2022-09-28 Created: 2022-09-28 Last updated: 2024-03-02Bibliographically approved
Kamat, J., Ortiz-Haro, J., Toussaint, M., Pokorny, F. T. & Orthey, A. (2022). BITKOMO: Combining Sampling and Optimization for Fast Convergence in Optimal Motion Planning. In: 2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS): . Paper presented at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), OCT 23-27, 2022, Kyoto, JAPAN (pp. 4492-4497). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>BITKOMO: Combining Sampling and Optimization for Fast Convergence in Optimal Motion Planning
Show others...
2022 (English)In: 2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 4492-4497Conference paper, Published paper (Refereed)
Abstract [en]

Optimal sampling based motion planning and trajectory optimization are two competing frameworks to generate optimal motion plans. Both frameworks have complementary properties: Sampling based planners are typically slow to converge, but provide optimality guarantees. Trajectory optimizers, however, are typically fast to converge, but do not provide global optimality guarantees in nonconvex problems, e.g. scenarios with obstacles. To achieve the best of both worlds, we introduce a new planner, BITKOMO, which integrates the asymptotically optimal Batch Informed Trees (BIT*) planner with the K-Order Markov Optimization (KOMO) trajectory optimization framework. Our planner is anytime and maintains the same asymptotic optimality guarantees provided by BIT*, while also exploiting the fast convergence of the KOMO trajectory optimizer. We experimentally evaluate our planner on manipulation scenarios that involve high dimensional configuration spaces, with up to two 7-DoF manipulators, obstacles and narrow passages. BITKOMO performs better than KOMO by succeeding even when KOMO fails, and it outperforms BIT* in terms of convergence to the optimal solution.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Series
IEEE International Conference on Intelligent Robots and Systems, ISSN 2153-0858
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-325028 (URN)10.1109/IROS47612.2022.9981732 (DOI)000908368203052 ()2-s2.0-85146319427 (Scopus ID)
Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), OCT 23-27, 2022, Kyoto, JAPAN
Note

QC 20230328

Available from: 2023-03-28 Created: 2023-03-28 Last updated: 2023-03-28Bibliographically approved
Poklukar, P., Polianskii, V., Varava, A., Pokorny, F. T. & Kragic, D. (2022). Delaunay Component Analysis for Evaluation of Data Representations. In: Proceedings 10th International Conference on Learning Representations, ICLR 2022: . Paper presented at 10th International Conference on Learning Representations, ICLR 2022, Apr 25-29, 2022 (online). International Conference on Learning Representations, ICLR
Open this publication in new window or tab >>Delaunay Component Analysis for Evaluation of Data Representations
Show others...
2022 (English)In: Proceedings 10th International Conference on Learning Representations, ICLR 2022, International Conference on Learning Representations, ICLR , 2022Conference paper, Published paper (Refereed)
Abstract [en]

Advanced representation learning techniques require reliable and general evaluation methods. Recently, several algorithms based on the common idea of geometric and topological analysis of a manifold approximated from the learned data representations have been proposed. In this work, we introduce Delaunay Component Analysis (DCA) - an evaluation algorithm which approximates the data manifold using a more suitable neighbourhood graph called Delaunay graph. This provides a reliable manifold estimation even for challenging geometric arrangements of representations such as clusters with varying shape and density as well as outliers, which is where existing methods often fail. Furthermore, we exploit the nature of Delaunay graphs and introduce a framework for assessing the quality of individual novel data representations. We experimentally validate the proposed DCA method on representations obtained from neural networks trained with contrastive objective, supervised and generative models, and demonstrate various use cases of our extended single point evaluation framework.

Place, publisher, year, edition, pages
International Conference on Learning Representations, ICLR, 2022
Keywords
Representation Learning, Machine Learning
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-312715 (URN)2-s2.0-85124640294 (Scopus ID)
Conference
10th International Conference on Learning Representations, ICLR 2022, Apr 25-29, 2022 (online)
Note

QC 20220614

Available from: 2022-05-20 Created: 2022-05-20 Last updated: 2023-09-07Bibliographically approved
Gieselmann, R. & Pokorny, F. T. (2022). Latent Planning via Expansive Tree Search. In: Advances in Neural Information Processing Systems 35 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022: . Paper presented at 36th Conference on Neural Information Processing Systems, NeurIPS 2022, New Orleans, United States of America, Nov 28 2022 - Dec 9 2022. Neural Information Processing Systems Foundation
Open this publication in new window or tab >>Latent Planning via Expansive Tree Search
2022 (English)In: Advances in Neural Information Processing Systems 35 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022, Neural Information Processing Systems Foundation , 2022Conference paper, Published paper (Refereed)
Abstract [en]

Planning enables autonomous agents to solve complex decision-making problems by evaluating predictions of the future. However, classical planning algorithms often become infeasible in real-world settings where state spaces are high-dimensional and transition dynamics unknown. The idea behind latent planning is to simplify the decision-making task by mapping it to a lower-dimensional embedding space. Common latent planning strategies are based on trajectory optimization techniques such as shooting or collocation, which are prone to failure in long-horizon and highly non-convex settings. In this work, we study long-horizon goal-reaching scenarios from visual inputs and formulate latent planning as an explorative tree search. Inspired by classical sampling-based motion planning algorithms, we design a method which iteratively grows and optimizes a tree representation of visited areas of the latent space. To encourage fast exploration, the sampling of new states is biased towards sparsely represented regions within the estimated data support. Our method, called Expansive Latent Space Trees (ELAST), relies on self-supervised training via contrastive learning to obtain (a) a latent state representation and (b) a latent transition density model. We embed ELAST into a model-predictive control scheme and demonstrate significant performance improvements compared to existing baselines given challenging visual control tasks in simulation, including the navigation for a deformable object.

Place, publisher, year, edition, pages
Neural Information Processing Systems Foundation, 2022
Series
Advances in Neural Information Processing Systems, ISSN 1049-5258 ; 35
National Category
Robotics and automation Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-331664 (URN)2-s2.0-85163176952 (Scopus ID)
Conference
36th Conference on Neural Information Processing Systems, NeurIPS 2022, New Orleans, United States of America, Nov 28 2022 - Dec 9 2022
Note

Part of ISBN 9781713871088

QC 20230712

Available from: 2023-07-13 Created: 2023-07-13 Last updated: 2025-02-05Bibliographically approved
Polianskii, V., Marchetti, G. L., Kravchenko, A., Varava, A., Pokorny, F. T. & Kragic, D. (2022). Voronoi Density Estimator for High-Dimensional Data: Computation, Compactification and Convergence. In: Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence: . Paper presented at The 38th Conference on Uncertainty in Artificial Intelligence, Eindhoven, The Netherlands, Aug 1-5 2022 (pp. 1644-1653). PMLR, 180
Open this publication in new window or tab >>Voronoi Density Estimator for High-Dimensional Data: Computation, Compactification and Convergence
Show others...
2022 (English)In: Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, PMLR , 2022, Vol. 180, p. 1644-1653Conference paper, Published paper (Refereed)
Abstract [en]

The Voronoi Density Estimator (VDE) is an established density estimation technique that adapts to the local geometry of data. However, its applicability has been so far limited to problems in two and three dimensions. This is because Voronoi cells rapidly increase in complexity as dimensions grow, making the necessary explicit computations infeasible. We define a variant of the VDE deemed Compactified Voronoi Density Estimator (CVDE), suitable for higher dimensions. We propose computationally efficient algorithms for numerical approximation of the CVDE and formally prove convergence of the estimated density to the original one. We implement and empirically validate the CVDE through a comparison with the Kernel Density Estimator (KDE). Our results indicate that the CVDE outperforms the KDE on sound and image data.

Place, publisher, year, edition, pages
PMLR, 2022
Series
Proceedings of Machine Learning Research, ISSN 2640-3498
National Category
Probability Theory and Statistics
Identifiers
urn:nbn:se:kth:diva-319195 (URN)2-s2.0-85163412377 (Scopus ID)
Conference
The 38th Conference on Uncertainty in Artificial Intelligence, Eindhoven, The Netherlands, Aug 1-5 2022
Note

QC 20221003

Available from: 2022-09-28 Created: 2022-09-28 Last updated: 2024-07-23Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-1114-6040

Search in DiVA

Show all publications