We present a framework for space partitioning, where the Regions of Influence (ROIs) of the agents are defined based on proximity metrics derived from the cost of optimal control problems. Efficient space partitioning in multi-agent systems, particularly in Unmanned Aerial Vehicle (UAV) operations, is critical for coverage, load balancing, and task allocation. However, traditional methods, such as the standard Voronoi Diagrams (VDs) based solely on distances, often fail to account for the dynamic behavior and capabilities of UAV s. We generalize the VD concept by replacing distance-based metrics with transition costs obtained from optimal control formulations. This allows the resulting partitions to incorporate UAV dynamics, including initial states and control effort, in defining regions where one agent is more suitable than another for a given task. We show that for a broad class of problems with second-order optimal costs, the boundaries between ROIs are given by either hyperplanes or quadratic surfaces. This includes, as special cases, classical VDs based on distance, minimum-time problems for single integrators, the fixed-final-state (FFS) optimal transfer problem, and Linear Quadratic Regulators (LQR). Overall, the proposed framework bridges geometric and control-theoretic space partitioning, enabling dynamic and context-aware task allocation in multi-agent systems.

Velocity Obstacles (VO) methods form a paradigm for collision avoidance strategies among moving obstacles and agents. While VO methods perform well in simple multi-agent environments, they do not guarantee safety and can show overly conservative behavior in common situations. In this paper, we propose to combine a VO strategy for guidance with a Control Barrier Function approach for safety, which overcomes the overly conservative behavior of VOs and formally guarantees safety. We validate our method in a baseline comparison study, using second-order integrator and car-like dynamics. Results support that our method outperforms the baselines with respect to path smoothness, collision avoidance, and success rates.

Beyond Visual Range (BVR) air combat is an essential part of modern aerial warfare, relying on advanced radar, missile systems, and decision-support technologies. This survey provides a comprehensive overview of how simulation and Machine Learning (ML) tools have been used to analyze BVR combat, covering key methodologies, applications, and challenges. We examine how ML enables adaptive tactics to improve behavior recognition and threat assessment to enhance situational awareness. The paper also traces the historical evolution of BVR combat, outlining key engagement phases such as detection, missile launch, and post-engagement assessment. A key focus is on the role of simulation environments in modeling realistic combat scenarios, supporting pilot training, and validating AI-driven decision-making strategies. We analyze state-of-the-art simulation tools, comparing their capabilities and limitations for studying multi-agent coordination and real-time adaptability. This survey's main contributions include descriptions of ML applications in BVR air combat, evaluations of simulation tools, identifications of research gaps, and insights into future research directions. This work provides an overview of how traditional simulation approaches merge with artificial intelligence to enable advanced, effective human and autonomous decision-making systems in dynamic and contested environments.

Developing new functionality for underwater robots and testing them in the real world is time-consuming and resource-intensive. Simulation environments allow for rapid testing before field deployment. However, existing tools lack certain functionality for use cases in our project: i) developing learning-based methods for underwater vehicles; ii) creating teams of autonomous underwater, surface, and aerial vehicles; iii) integrating the simulation with mission planning for field experiments. A holistic solution to these problems presents great potential for bringing novel functionality into the underwater domain. In this paper we present SMaRCSim, a set of simulation packages that we have developed to help us address these issues.

Tactical decisions in air combat are typically evaluated using experience as a basis. Pilots undergo frequent training in various air combat processes to enhance their combat proficiency and evaluation skills. Having the Situational Awareness (SA) necessary to evaluate the effects of multiple missile threats can often be challenging. This study provides a new method for calculating an aircraft fleet's maneuver flexibility in a Beyond-Visual-Range (BVR) setting. Sustaining a high degree of flexibility is necessary to adapt to unforeseen circumstances in BVR air combat. To do that, we employ Deep Neural Networks (DNN) to capture the result of a highperformance aircraft model in the presence of adversarial BVR missiles. We then modify our approach to calculate the aircraft's maneuverability concerning an opposing fleet, looking at the advantages and disadvantages of several flight formations. Finally, we consider the anticipated threat from an incoming opponent formation and optimize the counter-formation. This methodology offers a more sophisticated comprehension of aircraft maneuver flexibility within a BVR framework and aids in developing flexible and efficient decision-making techniques for air combat.

As the effective range of air-to-air missiles increases, it becomes harder for pilots and Unmanned aerial vehicle (UAV) operators to maintain the Situational Awareness (SA) needed to keep their aircraft safe. In this work, we propose a decision support tool to help pilots in Beyond Visual Range (BVR) air combat scenarios assess the risks of different options and make decisions based on those. Building upon earlier research that primarily addressed the threat of a single missile, we extend these ideas to encompass the complex scenario of multiple missile threats. The proposed method uses Deep Neural Networks (DNN) to learn from high-fidelity simulations and provide the pilots with an outcome estimate for a set of different strategies. Our results demonstrate that the proposed system can manage multiple incoming missiles, evaluate a family of options, and recommend the least risky course of action while accounting for all incoming air-to-air threats.

Behavior trees offer a modular approach to developing an overall controller from a set of sub-controllers that solve different sub-problems. These sub-controllers can be created using various methods, such as classical model-based control or reinforcement learning (RL). To achieve the overall goal, each sub-controller must satisfy the preconditions of the next sub-controller. Although every sub-controller may be locally optimal in achieving the preconditions of the next one, given some performance metric like completion time, the overall controller may still not be optimal with respect to the same performance metric. In this paper, we demonstrate how the performance of the overall controller can be improved if we use approximations of value functions to inform the design of a sub-controller of the needs of the next controller. We also show how, under certain assumptions, this leads to a globally optimal controller when the process is executed on all sub-controllers. Finally, this result also holds when some of the sub-controllers are already given. This means that if we are constrained to use some existing sub-controllers, the overall controller will be globally optimal, given this constraint.

Unmanned Aerial Systems (UAS) have the potential to provide cost effective solutions to many problems, but their control systems need to be safe and trustworthy in order to realize this potential. In this paper we show how behavior trees (BTs), created using backward chaining and using a particular way of labelling subtrees, can be used to meet the requirements of trustworthy autonomy described in a US air force (USAF) report. Behavior Trees represent a modular, reactive and transparent way of structuring a control system that is receiving increasing interest in the UAS community. While their safety and efficiency have been investigated in prior research, their connection to trustworthy autonomy has not been explored. A set of guidelines for trustworthy autonomy, taken from a USAF report, include items such as: being similar to how humans parse problems, being able to explain its reasoning in a concise way, and being able to be visualized at different levels of resolution. We propose a new way of deriving explanations that conform to these guidelines, using a particular labeling of subtrees in the BT combined with a structured design methodology called backward chaining. The proposed approach is illustrated in a detailed example.

Military aircraft pilots need to adjust to a constantly changing battlefield. A system that aids in understanding challenging combat circumstances and suggests appropriate responses can considerably improve the effectiveness of pilots. In this paper, we provide a Reinforcement Learning (RL) based system that acts as an aid in determining if an afterburner should be turned on to escape an incoming air-to-air missile. An afterburner is a component of a jet engine that increases thrust at the expense of exceptionally high fuel consumption. Thus it provides a short-term advantage, at the cost of a longterm disadvantage, in terms of reduced mission time. Helping to choose when to use the afterburner may significantly lengthen the flight duration, allowing aircraft to support friendly aircraft for longer and suffer fewer friendly fatalities due to this extended ability to provide support. We propose an RL-based risk estimation approach to help determine whether additional thrust is required to escape an incoming missile and study the benefits of thrust-aided evasive maneuvers. The suggested technique gives pilots a risk estimate for the scenario and a recommended course of action. We create an environment in which a pilot must decide whether or not to activate additional thrust to achieve the intended aim at a potentially high fuel consumption cost. Additionally, we investigate various trade-offs of the generated evasive maneuver policies.

In this paper we show how to improve the performance of backward chained behavior trees (BTs) that include policies trained with reinforcement learning (RL). BTs represent a hierarchical and modular way of combining control policies into higher level control policies. Backward chaining is a design principle for the construction of BTs that combines reactivity with goal directed actions in a structured way. The backward chained structure has also enabled convergence proofs for BTs, identifying a set of local conditions to be satisfied for the convergence of all trajectories to a set of desired goal states. The key idea of this paper is to improve performance of backward chained BTs by using the conditions identified in a theoretical convergence proof to configure the RL problems for individual controllers. Specifically, previous analysis identified so-called active constraint conditions (ACCs), that should not be violated in order to avoid having to return to work on previously achieved subgoals. We propose a way to set up the RL problems, such that they do not only achieve each immediate subgoal, but also avoid violating the identified ACCs. The resulting performance improvement depends on how often ACC violations occurred before the change, and how much effort, in terms of execution time, was needed to re-achieve them. The proposed approach is illustrated in a dynamic simulation environment.