The spurious transmission dips that occur in glide-symmetric holey gap waveguides (GSHGWs) are systematically characterized in this work, and the obtained information is used to suppress them in the intended operating band of the gap waveguide. The analysis relies on the dispersion characteristics of the waveguide segment with electromagnetic bandgap (EBG) holes. These characteristics are explored through the multimodal transfer matrix approach, particularly focusing on identifying relevant edge and waveguide modes. We find four types of unwanted dips in the transmission coefficient within the intended operation frequency band of the gap waveguide under study. The first three types are all associated with the edge mode mostly concentrated in the small air-gap region between the waveguide and the EBG holes, whereas the fourth type is caused by a narrow stopband in the waveguide mode. Based on a thorough understanding of all dips, we propose three viable solutions: placing EBG holes away from the waveguide channel, intersecting EBG holes with the waveguide channel, and intersecting additional small holes with the waveguide channel and the EBG holes. After comparison, the last solution with two small holes per EBG hole along the waveguide channel was demonstrated to be the most advantageous in terms of transmission properties, compactness, and flexibility. This solution was also experimentally validated using a WR-19 GSHGW operating from 35 to 63 GHz.