Preterm birth corresponds to a live birth before 37 weeks of gestation. It occurs in 5-15% of births worldwide and is becoming more common in almost every country. It is the primary cause of infant mortality and morbidity in both developed and developing countries. Additionally, it is associated with a higher mortality rate of 30-50% among young adult men and women. Late-onset sepsis corresponds to an infection of the blood stream of a neonate after 72 hours of life. It occurs in 15-25% of very-preterm infants, resulting in a 10% mortality rate and a threefold increase in morbidity. Detecting neonatal sepsis early and accurately could thus significantly reduce mortality, morbidity, and the use of antibiotics in premature infants. Predictors based upon deep architectures are often designed and evaluated in case control setups and using data derived from patient electrocardiogram (ECG) only. In an effort to bridge the gap between clinical and technical knowledge in neonatal sepsis detection research, we provide a detailed background of (1) the population under scrutiny, (2) the computation of features from vital signs signals, and (3) the supervised learning approach to neonatal sepsis detection. We then discuss a study evaluating deep recurrent architectures in a retrospective cohort study setup for neonatal sepsis detection. Data from different modalities were used, including chest impedance, pulse oximetry, ECG, demographics factors, and routine body weights measurements. The vanilla and long-short-term-memory (LSTM) Recurrent Neural Network (RNN) architectures were studied, and the performances were compared against logistic regression (LR) for a variety of classification metrics in a leave-one-out cross-validation framework. This study indicates that LSTM-based RNN models trigger less alarms than LR on a population of patients not suffering from sepsis. However, the performances in terms of precision and recall remain low, which indicates that further research is required before such models are implemented in clinical practice.