With the rapid development of the Internet of Things (IoT), the number of connected devices has increased exponentially, bringing significant convenience to various aspects of daily life and business operations. However, communication between IoT devices requires a significant amount of bandwidth, putting a strain on the communication system. To address this challenge, we introduce a classification-oriented semantic communication approach that transmits only essential information. We present a novel end-to-end task-oriented semantic communication model, which efficiently serves the classification task at the receiver. In particular, the proposed model first utilizes a neural network-based semantic encoder to extract classification-related semantic features. A transformer-based semantic decoder is used at the receiver to retrieve semantic features and generate classification results. We further introduce a channel encoder and decoder module to improve the ability of a single model to deal with various channel conditions. Simulation results show that, compared with the traditional method, the proposed scheme achieves higher classification accuracy on the ESC-50 dataset and UrbanSound8K dataset and has better performance for various channel conditions.