Ensuring user privacy while learning from the acquired Internet of Things sensor data, using limited available compute resources on edge devices, is a challenging task. Ideally, it is desirable to make all the features of the collected data private but due to resource limitations, it is not always possible as it may cause overutilization of resources, which in turn affects the performance of the whole system. In this work, we use the generalization techniques for data anonymization and provide customized injective privacy encoder functions to make data features private. Regardless of the resource availability, some data features must be essentially private. All other data features that may pose low privacy threat are termed as nonessential features. We propose Dynamic Iterative Greedy Search (DIGS), a novel approach with corresponding algorithms to select the set of optimal data features to be private for machine learning applications provided device resource constraints. DIGS selects the necessary and the most private version of data for the application, where all essential and a subset of nonessential features are made private on the edge device without resource overutilization. We have implemented DIGS in Python and evaluated it on Raspberry Pi model A (an edge device with limited resources) for an SVM-based classification on real-life health care data. Our evaluation results show that, while providing the required level of privacy, DIGS allows to achieve up to 26.21% memory, 16.67% CPU instructions, and 30.5% of network bandwidth savings as compared to making all the data private. Moreover, our chosen privacy encoding method has a positive impact on the accuracy of the classification model for our chosen application.