Chronic gastritis mainly includes chronic non-atrophic gastritis (CNAG), autoimmune gastritis (AIG), and type B gastritis. Early detection of AIG and type B gastritis will help identify high-risk groups for gastric cancer and prevent the development of irreversible peripheral neuropathy. We aim to develop a computer-assisted diagnosis (CADx) system by presenting a novel Convolution and Relative Self-Attention Parallel Network (CRSAPNet). We collected 3576 endoscopic images of chronic gastritis from 205 patients. MBConv and Relative Self-Attention Parallel Block (CRSAPB) was proposed to concatenate local features (such as mucosal folds and mucosal vessels extracted by MBConv) and global features (such as atrophied area extracted by Relative Self-Attention) in parallel in the last two stages of CRSAPNet. The CADx system distinguished AIG from type B gastritis and CNAG. The CRSAPNet achieved the highest overall accuracy of 95.44% (94.65% precision, 93.51% recall, 94.08% F1-score for AIG) with the fewest parameters. We used Grad-CAM to visually analyze the heat maps. We only replaced the original blocks of the third stage of ResNet50 and ConvNeXt-T with CRSAPB, resulting in an overall accuracy improvement of 0.37%, and 4.19%, respectively. Furthermore, the CADx system classified the three types of chronic gastritis for the first time. The CRSAPNet achieved an overall accuracy of 91.62%, and the overall accuracies in the location of the gastric body and gastric fundus were 93.43% and 92.51%, respectively. A new state-of-the-art deep learning network is introduced to distinguish AIG from type B gastritis and CNAG, and a classification for three types of chronic gastritis is reported for the first time.