To address the issue of low accuracy in speech fatigue recognition (SFR) under small samples, a method for small-sample SFR based on generative adversarial networks (GANs) is proposed. First, we enable the generator and discriminator to adversarially train and learn the features of the samples, and use the generator to generate high-quality simulated samples to expand our dataset. Then, we transfer discriminator parameters to fatigue identification network to accelerate network training speed. Furthermore, we use a bidirectional long short-term memory network (BLSTM) to further learn temporal fatigue features and improve the recognition rate of fatigue. 720 speech samples from a self-made Chinese speech database (SUSP-SFD) were chosen for training and testing. The results indicate that compared with traditional SFR methods, like convolutional neural networks (CNNs) and long short-term memory network (LSTM), our method improved the SFR rate by about 2.3-6.7%, verifying the effectiveness of the method.