Dynamic application security testing (DAST) scanning consists of automated requests to web applications with the goal of uncovering exploitable vulnerabilities. While the legitimate use of scanners aids development teams in improving security postures, they are often used by malicious actors in a brute-force manner for attack reconnaissance with a view to eventual compromise. Despite this threat from misuse of DAST to web applications and the critical data they handle, security mechanisms are lacking, with threshold-based classifiers suffering from being overly sensitive, causing excessive false positives. This paper demonstrates the first application of machine learning to specifically detect DAST attacks that augments a next-generation web application firewall implementing OWASP's AppSensor framework. Avoiding the brittle threshold approach and using tumbling windows of time to generate aggregated event features from source IPs, twelve random forest models are trained on millions of real-world events. Results show an optimal window size of 60 seconds achieves an F1 score of 0.94 and a miss rate of 6% on average across three production-grade web applications.