Estimating the state of an environment from high-dimensional, multimodal, and noisy observations is a fundamental challenge in reinforcement learning (RL). Traditional approaches rely on probabilistic models to account for the uncertainty, but often require explicit noise assumptions, in turn limiting generalization. In this work, we contribute a novel method to learn a structured latent representation, in which distances between states directly correlate with the minimum number of actions required to transition between them. The proposed metric space formulation provides a geometric interpretation of uncertainty without the need for explicit probabilistic modeling. To achieve this, we introduce a multimodal latent transition model and a sensor fusion mechanism based on inverse distance weighting, allowing for the adaptive integration of multiple sensor modalities without prior knowledge of noise distributions. We empirically validate the approach on a range of multimodal RL tasks, demonstrating improved robustness to sensor noise and superior state estimation compared to baseline methods. Our experiments show enhanced performance of an RL agent via the learned representation, eliminating the need of explicit noise augmentation. The presented results suggest that leveraging transition-aware metric spaces provides a principled and scalable solution for robust state estimation in sequential decision-making.

This thesis investigates how interaction can be used as self-supervision to learn structured state representations that simplify downstream tasks. We formalize two inductive biases naturally present in the trajectories generated by agents that interact with their environment: geometry and temporal consistency of the underlying state space. We show that injecting these biases into representation learning yields additional, taskrelevant properties. First, we focus on geometric bias: we learn translationally equivariant latent spaces from images in which agent actions correspond to vector additions. We show how these representations can be used to estimate a recovery policy that mitigates the compounding of error in data-driven sequential decision-making policies. We further extend equivariant representations to scenes with external objects. Under an interaction-by-contact model, we prove that aligning the object’s and the agent’s latent embeddings yields an isometric, disentangled representation of both. Second, we relax the geometry assumption and explore the milder temporal consistency bias. This allows us to construct representations where the temporal order between states is preserved, a property we refer to as distance monotonicity. In the reinforcement learning setting, we show that, under suitable conditions, this property is enough to recover an approximation of the value function and provably estimate an optimal policy. In a multiple-sensor framework, these representations can be used to construct a Bayesian filtering state estimate robust under unknown noise. Lastly, we extend the concept of interactions from physical systems to the parametric space of a learner. We show how distance monotonic representations of the parameters of a model can be used to approximate the posterior distribution of a Bayesian neural network. Finally,in a meta-learning setting, we explore implicit representations of the learner to reduce the variance of a fast-adaptation model. Collectively, these results demonstrate that interaction-driven biases produce structured representations that simplify or enhance the learning process.

Denna avhandling undersöker hur interaktion kan användas som självövervakning för att lära strukturerade tillståndsrepresentationer som förenklar nedströmsuppgifter. Vi formaliserar två induktiva bias som naturligt uppstår i trajektorier genererade av agenter som interagerar med sin omgivning: geometri samt temporal konsistens i det underliggande tillståndsrummet. Vi visar att införandet av dessa bias i representationsinlärning ger ytterligare, uppgiftsrelevanta egenskaper. Först fokuserar vi på geometrisk bias: vi lär translationsekvivarianta latenta rum från bilder där agentens handlingar motsvarar vektoradditioner. Vi visar hur sådana representationer kan användas för att estimera en återhämtningsstrategi som dämpar felackumulering i datadrivna, sekventiella beslutspolicys. Vi utvidgar därefter ekvivarianta representationer till scener med externa objekt. Under en kontaktbaserad interaktionsmodell bevisar vi att en inriktning (alignment) av objektets och agentens latenta inbäddningar ger en isometrisk och separerad (disentangled) representation av båda. Därefter lättar vi på geometriantagandet och studerar den mildare biasen temporal konsistens. Detta möjliggör konstruktion av representationer där den temporala ordningen mellan tillstånd bevaras—en egenskap vi benämner distansmonotonicitet. I en förstärkningsinlärningsmiljö visar vi att denna egenskap, under lämpliga villkor, räcker för att återvinna en approximation av värdefunktionen och bevisligen skatta en optimal policy. I ett flersensorramverk kan dessa representationer dessutom användas för att konstruera en Bayesiansk filtreringsbaserad tillståndsskattning som är robust mot okänt brus. Slutligen utvidgar vi interaktionsbegreppet från fysikaliska system till en lärares parametriska rum. Vi visar hur distansmonotona representationer av modellparametrar kan utnyttjas för att approximera posteriordistributionen i en Bayesiansk neuronnätmodell. I en meta-inlärningssättning undersöker vi även implicita representationer av läraren för att minska variansen hos en modell för snabb anpassning. Sammantaget demonstrerar resultaten att interaktionsdrivna bias leder till strukturerade representationer som förenklar eller förbättrar inlärningsprocessen.

Bayesian Neural Networks provide a principled framework for uncertainty quantification by modeling the posterior distribution of network parameters. However, exact posterior inference is computationally intractable, and widely used approximations like the Laplace method struggle with scalability and posterior accuracy in modern deep networks. In this work, we revisit sampling techniques for posterior exploration, proposing a simple variation tailored to efficiently sample from the posterior in over-parameterized networks by leveraging the low-dimensional structure of loss minima. Building on this, we introduce a model that learns a deformation of the parameter space, enabling rapid posterior sampling without requiring iterative methods. Empirical results demonstrate that our approach achieves competitive posterior approximations with improved scalability compared to recent refinement techniques. These contributions provide a practical alternative for Bayesian inference in deep learning.

Given a finite set of sample points, meta-learning algorithms aim to learn an optimal adaptation strategy for new, unseen tasks. Often, this data can be ambiguous as it might belong to different tasks concurrently. This is particularly the case in meta-regression tasks. In such cases, the estimated adaptation strategy is subject to high variance due to the limited amount of support data for each task, which often leads to sub-optimal generalization performance. In this work, we address the problem of variance reduction in gradient-based meta-learning and formalize the class of problems prone to this, a condition we refer to as task overlap. Specifically, we propose a novel approach that reduces the variance of the gradient estimate by weighing each support point individually by the variance of its posterior over the parameters. To estimate the posterior, we utilize the Laplace approximation, which allows us to express the variance in terms of the curvature of the loss landscape of our meta-learner. Experimental results demonstrate the effectiveness of the proposed method and highlight the importance of variance reduction in meta-learning.

We study the problem of learning graph dynamics of deformable objects that generalizes to unknown physical properties. Our key insight is to leverage a latent representation of elastic physical properties of cloth-like deformable objects that can be extracted, for example, from a pulling interaction. In this paper we propose EDO-Net (Elastic Deformable Object - Net), a model of graph dynamics trained on a large variety of samples with different elastic properties that does not rely on ground-truth labels of the properties. EDO-Net jointly learns an adaptation module, and a forward-dynamics module. The former is responsible for extracting a latent representation of the physical properties of the object, while the latter leverages the latent representation to predict future states of cloth-like objects represented as graphs. We evaluate EDO-Net both in simulation and real world, assessing its capabilities of: 1) generalizing to unknown physical properties, 2) transferring the learned representation to new downstream tasks.

Physical interaction with textiles, such as assistivedressing or household tasks, requires advanced dexterous skills.The complexity of textile behavior during stretching and pullingis influenced by the material properties of the yarn and bythe textile’s construction technique, which are often unknownin real-world settings. Moreover, identification of physicalproperties of textiles through sensing commonly available onrobotic platforms remains an open problem. To address this,we introduce Elastic Context (EC), a method to encode theelasticity of textiles using stress-strain curves adapted fromtextile engineering for robotic applications. We employ EC tolearn generalized elastic behaviors of textiles and examine theeffect of EC dimension on accurate force modeling of real-worldnon-linear elastic behaviors.

We address the problem of learning representations from observations of a scene involving an agent and an external object the agent interacts with. To this end, we propose a representation learning framework extracting the location in physical space of both the agent and the object from unstructured observations of arbitrary nature. Our framework relies on the actions performed by the agent as the only source of supervision, while assuming that the object is displaced by the agent via unknown dynamics. We provide a theoretical foundation and formally prove that an ideal learner is guaranteed to infer an isometric representation, disentangling the agent from the object and correctly extracting their locations. We evaluate empirically our framework on a variety of scenarios, showing that it outperforms vision-based approaches such as a state-of-the-art keypoint extractor. We moreover demonstrate how the extracted representations enable the agent to solve downstream tasks via reinforcement learning in an efficient manner.

Retrieval of mental images from measured brain activity may facilitate communication, especially when verbal or muscular communication is impossible or inefficient. The existing work focuses mostly on retrieving the observed visual stimulus while our interest is on retrieving the imagined mental image. We present a closed-loop BCI framework to retrieve mental images of human faces. We utilize EEG signals as binary feedback to determine the relevance of an image to the target mental image. We employ the feedback to traverse the latent space of a generative model to propose new images closer to the actual target image. We evaluate the proposed framework on 13 volunteers. Unlike previous studies, we do not restrict the possible attributes of the resulting images to predefined semantic classes. Subjective and objective tests validate the ability of our model to retrieve face images similar to the actual target mental images.

Learning from previously collected datasets of expert data offers the promise of acquiring robotic policies without unsafe and costly online explorations. However, a major challenge is a distributional shift between the states in the training dataset and the ones visited by the learned policy at the test time. While prior works mainly studied the distribution shift caused by the policy during the offline training, the problem of recovering from out-of-distribution states at the deployment time is not very well studied yet. We alleviate the distributional shift at the deployment time by introducing a recovery policy that brings the agent back to the training manifold whenever it steps out of the in-distribution states, e.g., due to an external perturbation. The recovery policy relies on an approximation of the training data density and a learned equivariant mapping that maps visual observations into a latent space in which translations correspond to the robot actions. We demonstrate the effectiveness of the proposed method through several manipulation experiments on a real robotic platform. Our results show that the recovery policy enables the agent to complete tasks while the behavioral cloning alone fails because of the distributional shift problem.

We study the problem of learning optimal behavior from sub-optimal datasets for goal-conditioned offline reinforcement learning under sparse rewards, invertible actions and deterministic transitions. To mitigate the effects of distribution shift, we propose MetricRL, a method that combines metric learning for value function approximation with weighted imitation learning for policy estimation. MetricRL avoids conservative or behavior-cloning constraints, enabling effective learning even in severely sub-optimal regimes. We introduce distance monotonicity as a key property linking metric representations to optimality and design an objective that explicitly promotes it. Empirically, MetricRL consistently outperforms prior state-of-the-art goal-conditioned RL methods in recovering near-optimal behavior from sub-optimal offline data.