Composite insulation systems usually exhibit nonlinear phenomena, such as voltage-dependent dielectric response and high-order current harmonics. In this article, a high-voltage system capable of performing both dielectric frequency response (DFR) and polarization/depolarization current (PDC) measurements up to 20 kV is devised. The nonlinear properties of the motor stator and power transformer insulation systems are studied. Experimental results show that the voltage-dependent effect becomes significant as the voltage magnitude approaches the nominal and frequency decreases. The third current harmonic component is found to be dominant over the other components and has a strong correlation with the decrease in the dissipation factor. By taking both the fundamental and third-harmonic components into account, more consistent voltage-dependent characteristics are revealed. These features are important indications of insulation defects that are often neglected in field diagnostic testings. The investigation of the nonlinear properties helps to enhance the condition diagnosis reliability.