Ethical hacking and vulnerability assessments are gaining rapid momentum as academic fields of study. Still, it is sometimes unclear what research areas are included in the categories and how they fit into the traditional academic framework. Previous studies have reviewed literature in the field, but the attempts use manual analysis and thus fail to provide a comprehensive view of the domain. To better understand how the area is treated within academia, 537,629 related articles from the Scopus database were analyzed. A Python script was used for data mining as well as analysis of the data, and 23,459 articles were included in the final synthesis. The publication dates of the articles ranged from 1975 to 2022. They were authored by 53,495 authors and produced an aggregated total of 836,956 citations. Fifteen research communities were detected using the Louvain community detection algorithm: (smart grids, attack graphs, security testing, software vulnerabilities, Internet of Things (IoT), network vulnerability, vulnerability analysis, Android, cascading failures, authentication, Software-Defined Networking (SDN), spoofing attacks, malware, trust models, and red teaming). In addition, each community had several individual subcommunities, constituting a total of 126. From the trends of the analyzed studies, it is clear that research interest in ethical hacking and vulnerability assessment is increasing.

Connected devices have become an integral part of modern homes and household devices, such as vac-uum cleaners and refrigerators, are now often connected to networks. This connectivity introduces an entry point for cyber attackers. The plethora of successful cyber attacks against household IoT indicates that the security of these devices, or the security of applications related to these devices, is often lacking. Existing penetration testing studies usually focus on individual devices, and recent studies often men-tion the need for more extensive vulnerability assessments. Therefore, this study investigates the cyber security of devices commonly located in connected homes. Systematic penetration tests were conducted on 22 devices in five categories related to connected homes: smart door locks, smart cameras, smart car adapters/garages, smart appliances, and miscellaneous smart home devices. In total, 17 vulnerabilities were discovered and published as new CVEs. Some CVEs received critical severity rankings from the National Vulnerability Database (NVD), reaching 9.8/10. The devices are already being sold and used worldwide, and the discovered vulnerabilities could lead to severe consequences for residents, such as an attacker gaining physical access to the house. In addition to the published CVEs, 52 weaknesses were discovered that could potentially lead to new CVEs in the future. To our knowledge, this is the most comprehensive study on penetration testing of connected household products.

AI programs, built using large language models, make it possible to automatically create phishing emails based on a few data points about a user. The V-Triad is a set of rules for manually designing phishing emails to exploit our cognitive heuristics and biases. In this study, we compare the performance of phishing emails created automatically by GPT-4 and manually using the V-Triad. We also combine GPT-4 with the V-Triad to assess their combined potential. A fourth group, exposed to generic phishing emails, was our control group. We use a red teaming approach by simulating attackers and emailing 112 participants recruited for the study. The control group emails received a click-through rate between 19-28%, the GPT-generated emails 30-44%, emails generated by the V-Triad 69-79%, and emails generated by GPT and the V-Triad 43-81%. Each participant was asked to explain why they pressed or did not press a link in the email. These answers often contradict each other, highlighting the importance of personal differences. Next, we used four popular large language models (GPT, Claude, PaLM, and LLaMA) to detect the intention of phishing emails and compare the results to human detection. The language models demonstrated a strong ability to detect malicious intent, even in non-obvious phishing emails. They sometimes surpassed human detection, although often being slightly less accurate than humans. Finally, we analyze of the economic aspects of AI-enabled phishing attacks, showing how large language models increase the incentives of phishing and spear phishing by reducing their costs.

As large language models continue to evolve, they have the potential to automate and enhance various aspects of computer security, including red teaming assessments. In this article, we conduct 32 computer security attacks and compare their success rates when performed manually and with assistance from large language models. The security assessments target five connected devices commonly found in modern households (two door locks, one vacuum cleaner, one garage door, and one smart vehicle adapter). We use attacks such as denial-of-service attacks, Man-in-the-Middle, authentication brute force, malware creation, and other common attack types. Each attack was performed twice, once by a human and once by an LLM, and scored for damage, reproducibility, exploitability, affected users, and discoverability based on the DREAD framework for computer security risk assessments. For the LLM-assisted attacks, we also scored the LLM's capacity to perform the attack autonomously. LLMs regularly increased the reproducibility and exploitability of attacks, but no LLM-based attack enhanced the damage inflicted on the device, and the language models often required manual input to complete the attack.