In this paper we present a safety argument fragment to contribute towards solutions to several key factors of relevance towards deployment of safe Automated Driving Systems (ADSs). Firstly, we address the need for exhaustive safety requirements by considering vehicle level, quantitative safety requirements. Secondly, situation awareness is employed to dynamically adapt the ADS’ decision-making. Thirdly, the ADS’ situation awareness is extended with constraints following Precautionary Safety (PcS) principles to ensure the fulfilment of the quantitative safety requirements. Fourthly, the models and assumptions supporting steps two and three are ascertained through the use of an operational design domain, which the ADS is designed to operate within. Furthermore, the paper contrasts the proposed argument with the state of the art in safety assurance to identify the key challenges still remaining.

By relieving the human driver of the responsibility of safely operating the vehicle, Automated Driving Systems (ADSs) (colloquially known as self-driving cars) can free up time and possibly also reduce the number of road accidents. Paradoxically, even though safety is one of the main expectations of ADSs, it is also one of the major challenges and arguably one of the key reasons why we have yet to see widespread market deployment of such systems. Contrary to previous generations of automotive systems, common development and safety assurance practises no longer suffice to accommodate the increased system complexity and operational uncertainty inherent to an ADS. Indeed, concrete models and means to show safety fulfilment before deployment remain elusive. For that purpose, this thesis focuses on efficient strategies for safety assurance of ADSs and explores this from three angles. 

Firstly, a comprehensive review of the state of the art has been conducted to identify and structure available methods for providing (predictive) evidence of the safety of the ADS, and to identify gaps and directions where further research is needed.

Secondly, the task of ensuring completeness of both the Verification and Validation (V&V) as well as the safety requirements of the ADS has been explored. The appropriate definition, formalisation and management of an Operational Design Domain (ODD) provide a means to ensure alignment between specification, testing and operations of the ADS – suggesting one way of closing the completeness gap for the V&V. Furthermore, to address the exhaustiveness of the safety requirements, this thesis proposes the use of a Quantitative Risk Norm (QRN) to elicit quantitative vehicle-level requirements. A QRN facilitates this exhaustiveness by considering frequencies of loss events (e.g. accidents) rather than requiring an enumeration of all possible hazards pertaining to the ADS.

Thirdly, this thesis extends the concept of Precautionary Safety (PcS) proposing a methodology for connecting the quantitative safety requirements of the QRN and the runtime decisions of the ADS. This is enabled by augmenting the ADS’s situation awareness (SAW) with an understanding of its own ability to avoid different loss events. Using this enhanced SAW model and by subsequently accounting for the uncertainties of the loss event probabilities, enables an assessment of the QRN even when there is limited data available. Consequently, the proposed methodology can ensure that the ADS indeed only takes decisions that are known to fulfil the QRN.

Jointly, the work presented in this thesis paves a way for how to bridge quantitative safety requirements and runtime decision-making of the ADS, and a possible strategy for efficient safety assurance of ADSs is outlined – drawing upon the contributions of the appended papers. There are still several open questions to understand the implications of this approach but the work showcased herein provides a solid foundation for such future work.

Automatiserade förarsystem (ADSer) (även kallade självkörande bilar) kan frigöra tid och möjligen även minska antalet olyckor i traffiken, genom att avlösa den mänskliga föraren från ansvaret för att köra säkert. Även om säkerhet (safety, security är inte inkluderat i denna avhandling) är en av de största förväntningarna på ADSer, så är det paradoxalt nog även en av de största utmaningarna. Kanske till och med en av huvudanledningarna till att vi ännu inte har sett någon bred lansering av denna typ av system på våra vägar. Metoder för utveckling och säkerhetsbevisning som använts för tidigare generationers system inom bilindustrin är inte längre tillräckliga för att hantera den ökade systemkomplexiteten och de osäkerhetsfaktorer som kännetecknar en ADS. Trots framsteg saknas accepterade, konkreta modeller och metoder för att framställa säkerhetsbevis innan ADSen lanseras på publika vägar. Som en del i att råda bot på detta fokuserar denna avhandling på strategier för säkerhetsbevisning av ADSer och utforskar detta område ur tre vinklar. 

För det första, har en omfattande litteraturestudie genomförts för att identifiera och strukturera befintliga metoder som bidrar till säkerhetsbevisningen för ADSer. I det arbetet identifierades också kvarstående forskningsluckor, som kräver ytterligare forskning.

För det andra, har komplettheten av både verifikationen och valideringen (V&V) samt säkerhetskraven på ADSen utforskats. Genom att bidra med en tillräcklig definition, formalisering och hantering av en Operational Design Domain (ODD) kan det verktyget stötta både specifikationen och testningen av systemet samt när det väl är i funktion (i runtime). ODDen ger således en potentiell väg framåt för att säkerställa komplettheten av V&V processerna och fyra konkreta strategier för att undvika att lämna ODDen presenteras. Vidare, så har en Kvantitativ Risk Norm (QRN) föreslagits för att förenkla arbetet med att uppnå kompletthet av säkerhetskraven på ADSen. Detta genom att kräva uppfyllnad av kvantitativa krav på antalet incidenter istället för att kräva en uppräkning av alla potentiella risker (hazards).

För det tredje, har konceptet med försiktig säkerhet (Precautionary safety) (PcS) vidare-utvecklats för att ge en konkret koppling mellan uppfyllnaden av en QRN och de beslut ADSen tar i runtime. Detta möjliggörs genom att utöka ADSens medvetenhet (situation awareness, SAW) om sin omgivning med en förståelse för det egna systemets förmåga att undvika olika incidenter. Trots begränsad tillgång till data möjliggör denna metod att ta fram en säker körpolicy som uppfyller QRNen genom att hantera de olika osäkerheterna i modellerna som underbygger PcS konceptet. Denna hantering gör det även möjligt att ADSen bara tar beslut som den vet kommer uppfylla QRNen.

Dessa tre områden utgör en möjlig väg framåt för en effektiv (efficient inte bara effektiv) strategi för säkerhetsbevisning för ADSer. Det finns visserligen mycket jobb kvar att göra för att förstå alla implikationer av denna strategi, men det arbete som läggs fram i denna avhandling ger en bra bas att stå på inför en fortsatt utforskning av denna eller ytterligare strategier för effektiv säkerhetsbevisning av ADSer.

In recent years, enormous investments in Automated Driving Systems (ADSs) have distinctly advanced ADS technologies. Despite promises made by several high profile auto-makers, it has however become clear that the challenges involved for deploying ADS have been drastically underestimated. Contrary to previous generations of automotive systems, common design, development, verification and validation methods for safety critical systems do not suffice to cope with the increased complexity and operational uncertainties of an ADS. Therefore, the aim of this paper is to provide an understanding of existing methods for providing safety evidence and, most importantly, identifying the associated challenges and gaps pertaining to the use of each method. To this end, we have performed a literature review, articulated around four categories of methods: design techniques, verification and validation methods, run-time risk assessment, and run-time (self-)adaptation. We have identified and present eight challenges, collectively distinguishing ADSs from safety critical systems in general, and discuss the reviewed methods in the light of these eight challenges. For all reviewed methods, the uncertainties of the operational environment and the allocation of responsibility for the driving task on the ADS stand-out as the most difficult challenges to address. Finally, a set of research gaps is identified, and grouped into five major themes: (i) completeness of provided safety evidence, (ii) improvements and analysis needs, (iii) safe collection of closed loop data and accounting for tactical responsibility on the part of the ADS, (iv) integration of AI/ML-based components, and (v) scalability of the approaches with respect to the complexity of the ADS.

Scenario-based approaches have been receiving a huge amount of attention in research and engineering of automated driving systems. Due to the complexity and uncertainty of the driving environment, and the complexity of the driving task itself, the number of possible driving scenarios that an Automated Driving System or Advanced Driving-Assistance System may encounter is virtually infinite. Therefore it is essential to be able to reason about the identification of scenarios and in particular critical ones that may impose unacceptable risk if not considered. Critical scenarios are particularly important to support design, verification and validation efforts, and as a basis for a safety case. In this paper, we present the results of a systematic mapping study in the context of autonomous driving. The main contributions are: (i) introducing a comprehensive taxonomy for critical scenario identification methods; (ii) giving an overview of the state-of-the-art research based on the taxonomy encompassing 86 papers between 2017 and 2020; and (iii) identifying open issues and directions for further research. The provided taxonomy comprises three main perspectives encompassing the problem definition (the why), the solution (the methods to derive scenarios), and the assessment of the established scenarios. In addition, we discuss open research issues considering the perspectives of coverage, practicability, and scenario space explosion.

Ensuring safety is arguably one of the largest remaining challenges before wide-spread market adoption of Automated Driving Systems (ADSs). One central aspect is how to provide evidence for the fulfilment of the safety claims and, in particular, how to produce a predictive and reliable safety case considering both the absence and the presence of faults in the system. In order to provide such evidence, there is a need for describing and modelling the different elements of the ADS and its operational context: models of event exposure, sensing and perception models, as well as actuation and closed-loop behaviour representations. This paper explores how estimates from such statistical models can impact the performance and operation of an ADS and, in particular, how such models can be continuously improved by incorporating more field data retrieved during the operation of (previous versions of) the ADS. Focusing on the safe driving velocity,  this results in the ability to update the driving policy so to maximise the allowed safe velocity, for which the safety claim still holds. For illustration purposes, an example considering statistical models of the exposure to an adverse event, as well as failures related to the system's perception system, is analysed. Estimations from these models, using statistical confidence limits, are used to derive a safe driving policy of the ADS. The results highlight the importance of leveraging field data in order to improve the system's abilities and performance, while remaining safe. The proposed methodology, leveraging a data-driven approach, also shows how the system's safety can be monitored and maintained, while allowing for incremental expansion and improvements of the ADS. 

More effective, efficient and flexible ways to manage safety assurance are needed for the successful development and release of Automated Driving Systems (ADSs). In this paper we propose a set of desired assurance method criteria and present an initial overview of available safety assurance methods and how they contribute to the proposed criteria. We observe that there is a significant gap between the state-of-the-art research and the state-of-practise for safety assurance of ADSs and propose to investigate reasons for this as future work. A next step will be to investigate how to merge the elements from the different assurance methods to achieve a method addressing all criteria. 

This is the data extraction form for the systematic literature review work for finding critical scenarios for automated driving systems. The extracted data from the primary studies is structured in the following tables. Primary studies in Tables 1 to 5 correspond to the five clusters defined in Section 6 of the main paper. Please note that some primary studies in these tables are classified as out of the scope of the literature study. These studies are marked in the Purpose column. Primary studies in Tables 6 and 7 are eventually considered as out of the scope. The tables are designed aligned with the taxonomy proposed in Section 4 of the main paper. 

We have yet to see wide deployment of automated driving systems (ADSs) on public roads. One of the reasons is the challenge of ensuring the systems' safety. The operational design domain (ODD) can be used to confine the scope of the ADS and subsequently also its safety case. For this to be valid the ADS needs to have strategies to remain in the ODD throughout its operations. In this paper we discuss the role of the minimal risk condition (MRC) as a means to ensure this. Further, we elaborate on the need for hierarchies of MRCs to cope with diverse system degradations during operations.

Automated Driving Systems (ADSs) show great potential to improve our transport systems. Safety validation, before market launch, is challenging due to the large number of miles required to gather enough evidence for a proven in use argumentation. Hence there is ongoing research to find more effective ways of verifying and validating the safety of ADSs. It is crucial both for the design as well as the validation to have a good understanding of the environment of the ADS. A natural way of characterizing the external conditions is by modelling and analysing data from real traffic. Towards this end, we present a framework with the primary ultimate objective to completely model and quantify the statistically relevant actions that other vehicles conduct on motorways. Two categories of fundamental actions are identified by recognising that a vehicle can only move longitudinally and laterally. The fundamental actions are defined in detail to create a set that is collectively exhaustive and mutually exclusive. All physically possible combinatorial actions that can be constructed from the fundamental actions are presented. To increase the granularity of the modelling the combinatorial actions are proposed to be analysed as sequences. Further, multi-vehicle interactions, which capture correlations between actions from multiple vehicles, are discussed. The resulting modularity of the framework allows for performing statistical analysis at an arbitrary granularity to support the design of a performant ADS as well as creating applicable validation scenarios. The use of the framework is demonstrated by automatically identifying fundamental actions in field data. Identified trajectories of two types of actions are visualised and the distributions for one parameter characterising each action type are estimated.

One of the major challenges of automated driving systems (ADS) is showing that they drive safely. Key to ensuring safety is eliciting a complete set of top-level safety requirements (safety goals). This is typically done with an activity called hazard analysis and risk assessment (HARA). In this paper we argue that the HARA of ISO 26262:2018 is not directly suitable for an ADS, both because the number of relevant operational situations may be vast, and because the ability of the ADS to make decisions in order to reduce risks will affect the analysis of exposure and hazards. Instead we propose a tailoring using a quantitative risk norm (QRN) with consequence classes, where each class has a limit for the frequency within which the consequences may occur. Incident types are then defined and assigned to the consequence classes; the requirements prescribing the limits of these incident types are used as safety goals to fulfil in the implementation. The main benefits of the QRN approach are the ability to show completeness of safety goals, and make sure that the safety strategy is not limited by safety goals which are not formulated in a way suitable for an ADS.