We study the capabilities of IceCube to search for sterile neutrinos with masses above 10 eV by analyzing its nu(mu) disappearance atmospheric neutrino sample. We find that IceCube is not only sensitive to the mixing of sterile neutrinos to muon neutrinos, but also to the more elusive mixing with tau neutrinos through matter effects. The currently released 1-year data shows a mild preference, between 0.75 and 3 sigma depending on the binning and flux adopted, for non-zero sterile mixing. This hint overlaps with the favored region for the sterile neutrino interpretation of the ANITA upward shower although the null results from CHORUS and NOMAD on nu(mu) to nu(tau) oscillations in vacuum exclude this interpretation, while through a different channel and using a different energy range. At the 99% C.L. an upper bound is obtained that improves over the present Super-Kamiokande and DeepCore constraints in some parts of the parameter space. We also investigate the physics reach of the roughly 8 years of data that is already on tape as well as a forecast of 20 years data to probe the present hint or improve upon current constraints.