Open this publication in new window or tab >>Show others...
2024 (English)In: KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery (ACM) , 2024, p. 2560-2571Conference paper, Published paper (Refereed)
Abstract [en]
Time Series Representation Learning (TSRL) focuses on generating informative representations for various Time Series (TS) modeling tasks. Traditional Self-Supervised Learning (SSL) methods in TSRL fall into four main categories: reconstructive, adversarial, contrastive, and predictive, each with a common challenge of sensitivity to noise and intricate data nuances. Recently, diffusion-based methods have shown advanced generative capabilities. However, they primarily target specific application scenarios like imputation and forecasting, leaving a gap in leveraging diffusion models for generic TSRL. Our work, Time Series Diffusion Embedding (TSDE), bridges this gap as the first diffusion-based SSL TSRL approach. TSDE segments TS data into observed and masked parts using an Imputation-Interpolation-Forecasting (IIF) mask. It applies a trainable embedding function, featuring dual-orthogonal Transformer encoders with a crossover mechanism, to the observed part. We train a reverse diffusion process conditioned on the embeddings, designed to predict noise added to the masked part. Extensive experiments demonstrate TSDE's superiority in imputation, interpolation, forecasting, anomaly detection, classification, and clustering. We also conduct an ablation study, present embedding visualizations, and compare inference speed, further substantiating TSDE's efficiency and validity in learning representations of TS data.
Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2024
Keywords
anomaly detection, classification, clustering, diffusion model, forecasting, imputation, interpolation, multivariate time series, representation learning, self-supervised learning, time series modeling
National Category
Computer graphics and computer vision Computer Sciences
Identifiers
urn:nbn:se:kth:diva-353962 (URN)10.1145/3637528.3671673 (DOI)001324524202061 ()2-s2.0-85203684729 (Scopus ID)
Conference
30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, Aug 25 2024 - Aug 29 2024
Note
Part of ISBN 9798400704901
QC 20240926
2024-09-252024-09-252025-03-17Bibliographically approved