Compared with the terrestrial networks (TN), which can only support limited coverage areas, low-earth orbit (LEO) satellites can provide seamless global coverage and high survivability in case of emergencies. Nevertheless, the swift movement of the LEO satellites poses a challenge: frequent handovers are inevitable, compromising the quality of service (QoS) of users and leading to discontinuous connectivity. Moreover, considering LEO satellite connectivity for different flying vehicles (FVs) when coexisting with ground terminals, an efficient satellite handover decision control and mobility management strategy is required to reduce the number of handovers and allocate resources that align with different users' requirements. In this paper, a novel distributed satellite handover strategy based on Multi-Agent Reinforcement Learning (MARL) and game theory named Nash- SAC has been proposed to solve these problems. From the simulation results, the Nash-SAC-based handover strategy can effectively reduce the handovers by over 16% and the blocking rate by over 18%, outperforming local benchmarks such as traditional Q-learning. It also greatly improves the network utility used to quantify the performance of the whole system by up to 48% and caters to different users' requirements, providing reliable and robust connectivity for both FVs and ground terminals.