For many practical applications, the polarization dependence of the Raman gain (Raman PDG) is considered as a drawback. Most approaches used to reduce the PDG are based on depolarization or scrambling of the pump source polarization. We report on an experimentally observed effect and describe a method for reducing the Raman PDG by means of polarization control with a fully polarized pump source. The method makes it possible to maintain high efficiency of the Raman gain. The absence of circular symmetry (on average) in non-polarization maintaining fibres allows one to find a certain polarization state of the pump to minimize the Raman PDG. The observed phenomenon is inherently related to polarization mode dispersion which always exists in optical fibres.