Wearable sensors are widely used in activity recognition (AR) tasks with broad applicability in health and well-being, sports, geriatric care, etc. Deep learning (DL) has been at the forefront of progress in activity classification with wearable sensors. However, most state-of-the-art DL models used for AR are trained to discriminate different activity classes at high accuracy, not considering the confidence calibration of predictive output of those models. This results in probabilistic estimates that might not capture the true likelihood and is thus unreliable. In practice, it tends to produce overconfident estimates. In this paper, the problem is addressed by proposing deep time ensembles, a novel ensembling method capable of producing calibrated confidence estimates from neural network architectures. In particular, the method trains an ensemble of network models with temporal sequences extracted by varying the window size over the input time series and averaging the predictive output. The method is evaluated on four different benchmark HAR datasets and three different neural network architectures. Across all the datasets and architectures, our method shows an improvement in calibration by reducing the expected calibration error (ECE)by at least 40%, thereby providing superior likelihood estimates. In addition to providing reliable predictions our method also outperforms the state-of-the-art classification results in the WISDM, UCI HAR, and PAMAP2 datasets and performs as good as the state-of-the-art in the Skoda dataset. 

Temporal patterns are encoded within the time-series data, and neural networks, with their unique feature extraction ability, process those patterns to provide a better predictive response. Ensembles of neural networks have proven to be very effective Human Activity Recognition (HAR) tasks with time-series data, e.g., wearable sensors. The combination of predictions coming from the individual models in the ensemble helps boost the overall classification metric through efficient temporal pattern recognition. Currently, the most common strategy for combining the predictions coming from the individual models is simple averaging. However, since each ensemble model learns different temporal patterns of the timseries classification problem, a simple averaging strategy is sub-optimal. This sub-optimality is addressed in this paper through a neural network-based adaptive learning framework. The method's core is training a neural gate that ingests the same input time-series data fed to the other temporal models. The goal of the training process is to adaptively learn scaler values against each temporal model by looking at the input data. These scaler values weigh each temporal model while combining the ensemble. The framework obtains superior predictive performance as compared to the standard ensembling techniques. The framework is evaluated on a benchmark HAR dataset called PAMAP2 [3] with two popular state-of-the-art ensemble architectures namely DTE [1] and LSTM-ensemble [2]. In both cases, the classification performance of the framework in HAR tasks surpasses the state-of-the-art models.

With the growing interest of the research community in making deep learning (DL) robust and reliable, detecting out-of-distribution (OOD) data has become critical. Detecting OOD inputs during test/prediction allows the model to account for discriminative features unknown to the model. This capability increases the model's reliability since this model provides a class prediction solely at incoming data similar to the training one. OOD detection is well established in computer vision problems. However, it remains relatively under-explored in other domains such as time series (i.e., Human Activity Recognition (HAR)). Since uncertainty has been a critical driver for OOD in vision-based models, the same component has proven effective in time-series applications. We plan to address the OOD detection problem in HAR with time-series data in this work. To test the capability of the proposed method, we define different types of OOD for HAR that arise from realistic scenarios. We apply an ensemble-based temporal learning framework that incorporates uncertainty and detects OOD for the defined HAR workloads. In particular, we extract OODs from popular benchmark HAR datasets and use the framework to separate those OODs from the indistribution (ID) data. Across all the datasets, the ensemble framework outperformed the traditional deep-learning method (our baseline) on the OOD detection task.

As the research community focuses on improving the reliability of deep learning, identifying out-of-distribution (OOD) data has become crucial. Detecting OOD inputs during test/prediction allows the model to account for discriminative features unknown to the model. This capability increases the model's reliability since this model provides a class prediction solely at incoming data similar to the training one. Although OOD detection is well-established in computer vision, it is relatively unexplored in other areas, like time series-based human activity recognition (HAR). Since uncertainty has been a critical driver for OOD in vision-based models, the same component has proven effective in time-series applications. In this work, we propose an ensemble-based temporal learning framework to address the OOD detection problem in HAR with time-series data. First, we define different types of OOD for HAR that arise from realistic scenarios. Then we apply our ensemble-based temporal learning framework incorporating uncertainty to detect OODs for the defined HAR workloads. This particular formulation also allows a novel approach to fall detection. We train our model on non-fall activities and detect falls as OOD. Our method shows state-of-The-Art performance in a fall detection task using much lesser data. Furthermore, the ensemble framework outperformed the traditional deep-learning method (our baseline) on the OOD detection task across all the other chosen datasets.

Differential privacy (DP) is a method to protect individual privacy when the data is used for downstream analytical tasks. The core ability of DP to quantity privacy numerically separates it from other privacy-preserving methods. In human activity recognition (HAR), differential privacy can protect users’ privacy who contribute their data to train machine learning algorithms. While some methods are developed for privacy protection in such cases, no method quantifies privacy and seamlessly integrates into machine learning frameworks like DP. The paper proposes a DP framework called TEMPDIFF (short for temporal differential privacy), which guarantees privacy preserving human activity recognition for wearable time-series data with competitive classification performance and works with any machine-learning/deep-learning methods. TEMPDIFF capitalizes on the temporal characteristics of wearable sensor data to improve the modelling task, which enhances the privacy-utility tradeoff. TEMPDIFF uses ensembling and a novel temporal partitioning algorithm for time-series data to ensure optimal training of ensemble models. In TEMPDIFF, consensus through ensembling and the addition of controlled Laplacian noise obscures sensitive information used to train the models, guaranteeing strict levels of differential privacy. The proposed method is evaluated on two popular HAR datasets. It outperforms the classification accuracy and privacy budget for both datasets compared to the state-of-the-art approaches.

Federated learning (FL), a decentralized machine learning technique, enhances privacy by enabling multiple devices to collaboratively train a model without transferring data to a central server. FL is used in Human Activity Recognition (HAR) problems, where multiple users generating private wearable data share models with a server to learn a useful global model. However, FL may compromise data privacy through model information sharing during training. Moreover, it adheres to a one-size-fits-all approach toward data privacy, potentially neglecting varied user preferences in collaborative scenarios such as HAR. In response to these challenges, this paper presents a collaborative learning framework integrating differential privacy (DP) and FL, thus providing a tailored approach to privacy protection. While some existing works integrate DP and FL, they do not allow clients to have different privacy preferences. In this work, we introduce a framework that allows different clients to have different privacy preferences and hence more flexibility in terms of privacy. In our framework, DP adds individualized noise to individual clients’ gradient updates for privacy. However, such noised updates can also be interpreted as an attack on the FL system. Defending against these attacks might result in excluding honest private clients altogether from training, posing a fairness concern. On the other hand, not having any defensive measures might allow malicious users to attack the system, posing a security issue. Thus, to address security and fairness, our framework incorporates a client selection strategy that protects the global model from malicious clients and provides fair model access to honest private clients. We have demonstrated the effectiveness of our system on a HAR dataset and provided insights into our framework’s privacy, utility, and fairness.

Federated learning (FL) stands as a crucial method in preserving the data privacy of individuals who actively contribute to the machine learning task of Human Activity Recognition (HAR) through wearable devices.Although FL provides a degree of privacy protection, it's crucial to acknowledge that it may not always guarantee robust models and data privacy in specific scenarios. Moreover, FL typically ensures privacy by keeping data localized within user infrastructure, offering a uniform perspective on data privacy.To tackle these challenges, researchers are increasingly focusing on integrating privacy-preserving techniques like Differential Privacy (DP), Homomorphic Encryption (HE), and Trusted Execution Environments (TEE) into the FL framework. However, integrating these techniques directly can introduce their own set of challenges.For instance, DP, while effective in preserving privacy, can disrupt the learning process if excessive noise is added, which is particularly challenging in FL due to data heterogeneity. HE offers strong privacy guarantees but is constrained by computational complexity, and TEEs face scalability issues in practical implementations.To strike a balance between privacy and utility, we introduce the \emph{Kavach} framework designed to seamlessly integrate specific privacy-preserving methods while accommodating diverse privacy preferences. In this framework, privacy is recognized as a non-uniform concept, where privacy does not arise only by withholding data but also by incorporating different privacy preferences and privacy-preserving methods. With respect to the non-uniformity, the quantification of privacy by users plays a crucial role.Kavach leverages DP's unique ability to quantify privacy and tailor privacy budgets to individual user needs. Additionally, the framework allows integration of other privacy methods like encryption or TEEs on a limited scale, reserving them for specific requirements.The primary goal of \textit{Kavach} is to provide a personalized privacy experience for both clients and system designers. It offers flexibility, enabling users to choose their preferred privacy settings and methods, ensuring a customized and effective approach to privacy preservation within collaborative learning.In this work, we present different variants of the \textit{Kavach} framework and showcase experiments on benchmark Human Activity Recognition (HAR) datasets, as well as in the computer vision domain. These experiments highlight the privacy-utility tradeoff and classification performance, demonstrating the framework's effectiveness in achieving personalized privacy while maintaining utility.