Older versions of the ROUGEeval summarization evaluation system were easier to fool
2007 (English)In: Information Processing & Management, ISSN 0306-4573, Vol. 43, no 6, 1500-1505 p.Article in journal (Refereed) Published
We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text. By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however not be considered good by human readers. The method can be adapted to trick different settings of the ROUGEeval package.
Place, publisher, year, edition, pages
2007. Vol. 43, no 6, 1500-1505 p.
automatic summarization, automatic evaluation, Markov models
Computer and Information Science
IdentifiersURN: urn:nbn:se:kth:diva-37195DOI: 10.1016/j.ipm.2007.01.014ISI: 000249742500006ScopusID: 2-s2.0-34547933293OAI: oai:DiVA.org:kth-37195DiVA: diva2:432523