Floris Goerlandt: Risk assessment validity in the MASS Code

Maritime Autonomous Surface Ships (MASS) continue to attract significant interest in industrial, regulatory, and academic environments. Aiming to revolutionize shipping by enabling maritime operations with reduced or minimal human intervention, these vessels rely on sensors, real-time communication, artificial intelligence, and automation, thereby enhancing efficiency, safety, and sustainability. Testbed applications at various degrees of autonomy have been piloted for several ship types, with commercial operations spearheaded in some markets. These include fully autonomous container ships in coastal navigation and remotely controlled inland vessels. Remote pilotage, where licensed pilots provide decision support for local navigation conditions from a remote location, has also received interest in several jurisdictions.

To enable safe, secure, and environmentally sound MASS operations, work is ongoing at the International Maritime Organization (IMO) to develop an International Code of Safety for Maritime Autonomous Surface Ships (MASS Code), which will supplement existing maritime regulations for aspects specific to MASS not covered under existing rules. It is designed as a goal-based instrument, in which high-level goals are formulated for a vessel, with Functional Requirements (FRs) and Expected Performance conditions (EPs) translating these goals into actionable benchmarks. These define what a MASS or its subsystems must achieve without prescribing specific methods or technologies, thus allowing flexibility and innovation.

In Goal-Based Standards, risk assessment is used to identify hazards, analyze and evaluate associated risks, and to support decisions on mitigating measures. Risk assessment can be used to identify and formulate FRs and EPs to operationalize the high-level goals of the generic MASS Code, and to ascertain that the technical design and operational procedures of a specific MASS meet these FRs and EPs. Various guidance documents exist for performing risk assessments in the maritime industry. For the MASS Code, IMO’s guidelines for Formal Safety Assessment and the Risk Based Assessment Tool being developed under the auspices of the European Maritime Safety Agency are arguably the most important.

Based on the MASS Code draft version of September 2024 (MSC 109/5), and relying on recent findings in risk and safety research, two issues warranting more focus in developing this regulatory instrument are highlighted here.

First, whereas the draft Code refers to the need to use “suitable, recognized, and appropriate risk assessment techniques”, there is little attention to the applicability of specific hazard and risk analysis methods for MASS systems. However, much academic work suggests that techniques such as Hazard and Operability (HAZOP) studies, Failure Mode and Effects Analysis (FMEA), and fault and event trees, all based on linear accident causation models, are ill-suited for assessing risk in complex sociotechnical systems. Considering for instance that MASS design and operation requires decision-making of many actors at various organizations and the importance of feedback mechanisms in the interconnected systems, which are essential tenets of accident causation in complex sociotechnical systems, techniques such as the Systems-Theoretic Process Analysis and the Event Analysis of Systemic Teamwork-Broken Links method are more suited for MASS risk assessment. These better align with systems views on accident causation, while research suggests that these identify more hazards than HAZOP and FMEA.

Second, the draft Code does not address quality requirements for MASS risk assessments, apart from tentatively noting that “risk assessment should be carried out by personnel with relevant expertise”. Research however suggests that the comprehensiveness of hazard identification can vary significantly and that important differences in risk ratings may arise when analyses are executed by different teams and with different techniques. Hence, given the pivotal role of risk assessment in the MASS Code, there should be clearer guidance for personnel in industry and administrations on risk assessment quality to ensure that analyses are comprehensive, accurate, and credible. Guidance such as the Risk Analysis Quality Test by the Society for Risk Analysis, as well as validation frameworks tailored to specific hazard and risk analysis techniques proposed in the academic literature, can provide fruitful starting points for this work.

If risk assessment validity is insufficiently considered, there is a risk of false assurance about safety, i.e. a belief that the risks are appropriately assessed and mitigated, whereas this confidence in meeting the safety goals is unwarranted. While developing a MASS Code is a nuanced exercise requiring inputs from various experts, advocates of evidence-based policy making are urged to take note of the above, to ensure a robust and sustainable path towards vessel autonomy.