Artificial intelligence (AI) holds significant promise as a technology that may improve the quality of educational practices. This includes specialized AI-powered technologies tailored for education and general AI-based technologies, including recently popular generative AI tools that stakeholders are increasingly adapting for teaching and learning. Integrating AI tools into educational settings holds numerous potential pedagogical benefits, such as assisting teachers in planning lessons, promoting personalization, and enhancing student autonomy. However, concerns about bias and discrimination linked to the use of these technologies have rapidly emerged. Today, standardized evaluation criteria to assess the potential contribution of such tools to education and their reliability within the learning and teaching context are lacking. To address this gap, we build on an existing taxonomy for the evaluation of open educational resources (OER) to better suit the unique features of generative AI. The result is a six-dimensional evaluation approach that includes descriptive, pedagogical, representational, communication, scientific content, as well as the ethical and transparency dimension. We then apply this approach to examine the educational potential and ethical concerns around 30 AI tools. The analysis facilitates a critical mapping of the potential and risks of AI-powered technologies in education settings.