This article considers the pursuit-evasion game of two dynamic systems, which are subject to safety constraints, and in order to additionally guarantee the safety of the system, we propose safety-aware pursuit and escape strategies by combining control barrier function (CBF) and off-policy learning technique. Different from existing pursuit and evader strategies, a safeguarding control law is first designed based on CBF to prioritize the safety of pursuer's and evader's trajectories, and then bounded game strategies are proposed by elaborately designing a new cost function. We also provide the sufficient condition for the stability of the closed-loop system with the state denoted by position difference, under which, the pursuer is able to capture the evader. It is worth mentioning that our strategies do not require the knowledge of system dynamics, which are essentially online learning-based ones, featured with the ability of satisfying the safety constraints in the pursuit-evasion game.