Previous work has shown that speech disfluencies can negatively impact judgments about a speaker's competence and confidence. However, these effects have primarily been examined with Likert-type rating scales, which are not informative about how judgments might translate to behavior. Does the presence of disfluencies actually guide decision-making when listeners stand to gain concretely from making the correct choice? We sought to address this question with a web-based decision task in which participants were asked to choose between two conflicting sources of information. Our results suggest that listeners do take speech fluency into account when deciding who or what to believe.