A fundamental challenge in mass spectrometry-based proteomics is determining which peptide generated a given MS2 spectrum. Peptide sequencing typically relies on matching spectra against a known sequence database, which in some applications is not available. Deep learning-based de novo sequencing can address this limitation by directly predicting peptide sequences from MS2 data. We have seen the application of the transformer architecture to de novo sequencing produce state-of-the-art results on the so-called nine-species benchmark. In this study, we propose an improved transformer encoder inspired by the heuristics used in the manual interpretation of spectra. We modify the attention mechanism with a learned bias based on pairwise mass differences, termed Pairwise Attention (PA). Adding PA improves average peptide precision at 100% coverage by 12.7% (5.9 percentage points) over our base transformer on the original nine-species benchmark. We have also achieved a 7.4% increase over the previously published model Casanovo. Our MS2 encoding strategy is largely orthogonal to other transformer-based models encoding MS2 spectra, enabling straightforward integration into existing deep-learning approaches. Our results show that integrating domain-specific knowledge into transformers boosts de novo sequencing performance.

Self-supervised learning provides a way to extract structure from tandem mass spectra without relying on peptide labels, which are typically obtained from database-search pipelines and therefore inherit their assumptions and biases. This work evaluates several self-supervised objectives: masked spectrum modeling, masked autoencoding, trinary m/zperturbation, and DINO-style self-distillation, using a shared transformer encoder trained on large unlabeled MS2 corpora. We analyze their training behavior, document characteristic failure modes such as collapse in DINO, and measure their effect on downstream de novo peptide sequencing and auxiliary prediction tasks.

Across controlled fine-tuning settings, masked autoencoding yields the most consistent improvements in de novo accuracy, with measurable gains even after very limited pretraining. DINO provides modest but reproducible improvements over scratch for de novo decoding and strong gains on global tasks, whereas the trinary perturbation objective produces only small and often inconsistent benefits. These results demonstrate that unsupervised objectives can recover meaningful structure from raw spectra, although the absolute de novo accuracies achieved here lie below those of state-of-the-art supervised systems, meaning the observed gains should be interpreted primarily as an initialization ablation rather than an indication of absolute model capability.

Overall, the study shows that self-supervision can influence MS2 representations in useful ways, clarifies which objectives are effective for current transformer architectures, and highlights the need for MS-specific pretraining tasks that more directly support high-quality sequence reconstruction