Unsupervised evaluation of parser robustness
2005 (English)In: COMPUTATIONAL LINGUISTICS AND INTELLIGENT TEXT PROCESSING / [ed] Gelbukh, A, 2005, Vol. 3406, 142-154 p.Conference paper (Refereed)
This article describes an automatic evaluation procedure for NLP system robustness under the strain of noisy and ill-formed input. The procedure requires no manual work or annotated resources. It is language and annotation scheme independent and produces reliable estimates on the robustness of NLP systems. The only requirement is an estimate on the NLP system accuracy. The procedure was applied to five parsers, and one part-of-speech tagger on Swedish text. To establish the reliability of the procedure, a comparative evaluation involving annotated resources was carried out on the tagger and three of the parsers.
Place, publisher, year, edition, pages
2005. Vol. 3406, 142-154 p.
, LECTURE NOTES IN COMPUTER SCIENCE, ISSN 0302-9743 ; 3406
IdentifiersURN: urn:nbn:se:kth:diva-42741ISI: 000228725100014ScopusID: 2-s2.0-24344460570ISBN: 3-540-24523-5OAI: oai:DiVA.org:kth-42741DiVA: diva2:447968
6th Annual Conference on Intelligent Text Processing and Computational Linguistics Location: Mexico City, MEXICO Date: FEB 13-19, 2005
QC 201110132011-10-132011-10-122011-10-13Bibliographically approved