Nano-FTIR spectroscopy is a technique where atomic force microscopy (AFM) and infrared (IR) spectroscopy are combined to obtain chemical information with a lateral resolution of some tens of nm. It has been used to study numerous solid surfaces and recently also liquids including water have been examined by separating the liquid from the AFM tip by a thin lid. However, although the water stretching vibrations are significantly more intense than the bending vibration in conventional IR spectroscopy, only the bending vibration has been observed in nano-FTIR spectroscopy so far. In this article we show that also the stretching vibrations of liquid H2O, D2O, HOD, and aqueous salt solutions can be probed in nano-FTIR spectroscopy. Nano-FTIR absorption and phase spectra have been acquired at different harmonics and it was found the third optical harmonic (O3) in the nano-FTIR absorption spectra exhibited the highest resemblance with attenuated total reflection (ATR). Being able to probe water stretching vibrations with nano-FTIR spectroscopy is of importance since the stretching vibrations contain considerably more detailed information regarding for example hydrogen bonding strength than the bending vibration. In addition, it enables studies of the essential interactions between water and biomolecules. Furthermore, this work highlights both advantages and challenges that nano-FTIR spectroscopy in a liquid sample cell has and importance of further studies that lead to a better understanding of the near-field signals in liquids.