Digital Library

Responsible Use of AI in Military Systems (Jan Maarten Schraagen) (z-library.sk, 1lib.sk, z-lib.sk)

Jan Maarten Schraagen

Share E-Book
Read Without Registration

Responsible Use of AI in Military Systems (Jan Maarten Schraagen) (z-library.sk, 1lib.sk, z-lib.sk)

Author Jan Maarten Schraagen

AI
Language English

Artificial Intelligence (AI) is widely used in society today. The (mis)use of biased data sets in machine learning applications is well‑known, resulting in discrimination and exclusion of citizens. Another example is the use of non‑transparent algorithms that can’t explain themselves to users, resulting in the AI not being trusted and therefore not being used when it might be beneficial to use it. - Responsible Use of AI in Military Systems lays out what is required to develop and use AI in military systems in a responsible manner. Current developments in the emerging field of Responsible AI as applied to military systems in general (not merely weapons systems) are discussed. The book takes a broad and transdisciplinary scope by including contributions from the fields of philosophy, law, human factors, AI, systems engineering, and policy development. - Divided into five sections, Section I covers various practical models and approaches to implementing military AI responsibly; Section II focuses on liability and accountability of individuals and states; Section III deals with human control in human‑AI military teams; Section IV addresses policy aspects such as multilateral security negotiations; and Section V focuses on ‘autonomy’ and ‘meaningful human control’ in weapons systems. - Key Features: • Takes a broad transdisciplinary approach to responsible AI • Examines military systems in the broad sense of the word • Focuses on the practical development and use of responsible AI • Presents a coherent set of chapters, as all authors spent two days discussing each other’s work - This book provides the reader with a broad overview of all relevant aspects involved with the responsible development, deployment and use of AI in military systems. It stresses both the advantages of AI as well as the potential downsides of including AI in military systems.

Format PDF
Size 6.2 MB
4
Views
0
Downloads
0.00
Total Donations
(First 20 pages)

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Page 1
(This page has no text content)
Page 2
Responsible Use of AI in Military Systems Artificial Intelligence (AI) is widely used in society today. The (mis)use of biased data sets in machine learning applications is well‑known, resulting in discrimina‑ tion and exclusion of citizens. Another example is the use of non‑transparent algo‑ rithms that can’t explain themselves to users, resulting in the AI not being trusted and therefore not being used when it might be beneficial to use it. Responsible Use of AI in Military Systems lays out what is required to develop and use AI in military systems in a responsible manner. Current developments in the emerging field of Responsible AI as applied to military systems in general (not merely weapons systems) are discussed. The book takes a broad and transdisci‑ plinary scope by including contributions from the fields of philosophy, law, human factors, AI, systems engineering, and policy development. Divided into five sections, Section I covers various practical models and approaches to implementing military AI responsibly; Section II focuses on liability and accountability of individuals and states; Section III deals with human control in human‑AI military teams; Section IV addresses policy aspects such as multilateral security negotiations; and Section V focuses on ‘autonomy’ and ‘meaningful human control’ in weapons systems. Key Features: • Takes a broad transdisciplinary approach to responsible AI • Examines military systems in the broad sense of the word • Focuses on the practical development and use of responsible AI • Presents a coherent set of chapters, as all authors spent two days discuss‑ ing each other’s work This book provides the reader with a broad overview of all relevant aspects involved with the responsible development, deployment and use of AI in military systems. It stresses both the advantages of AI as well as the potential downsides of including AI in military systems.
Page 3
Chapman & Hall/CRC Artificial Intelligence and Robotics Series Series Editor: Roman Yampolskiy Digital Afterlife and the Spiritual Realm Maggi Savin‑Baden A First Course in Aerial Robots and Drones Yasmina Bestaoui Sebbane AI by Design: A Plan for Living with Artificial Intelligence Catriona Campbell The Global Politics of Artificial Intelligence Edited by Maurizio Tinnirello Unity in Embedded System Design and Robotics: A Step‑by‑Step Guide Ata Jahangir Moshayedi, Amin Kolahdooz, and Liao Liefa Meaningful Futures with Robots: Designing a New Coexistence Edited by Judith Dörrenbächer, Marc Hassenzahl, Robin Neuhaus, and Ronda Ringfort‑Felner Topological Dynamics in Metamodel Discovery with Artificial Intelligence: From Biomedical to Cosmological Technologies Ariel Fernández A Robotic Framework for the Mobile Manipulator: Theory and Application Nguyen Van Toan and Phan Bui Khoi AI in and for Africa: A Humanist Perspective Susan Brokensha, Eduan Kotzé, and Burgert A. Senekal Artificial Intelligence on Dark Matter and Dark Energy: Reverse Engineering of the Big Bang Ariel Fernández Explainable Agency in Artificial Intelligence: Research and Practice Silvia Tulli and David W. Aha An Introduction to Universal Artificial Intelligence Marcus Hutter, Elliot Catt, and David Quarel AI: Unpredictable, Unexplainable, Uncontrollable Roman V. Yampolskiy Transcending Imagination: Artificial Intelligence and the Future of Creativity Alexander Manu Responsible Use of AI in Military Systems Jan Maarten Schraagen For more information about this series please visit: https://www.routledge.com/ Chapman‑‑HallCRC‑Artificial‑Intelligence‑and‑Robotics‑Series/book‑series/ ARTILRO
Page 4
Responsible Use of AI in Military Systems Edited by Jan Maarten Schraagen
Page 5
First edition published 2024 by CRC Press 2385 NW Executive Center Drive, Suite 320, Boca Raton FL 33431 and by CRC Press 4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN CRC Press is an imprint of Taylor & Francis Group, LLC © 2024 selection and editorial matter, Jan Maarten Schraagen; individual chapters, the contributors Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. The Open Access version of this book, available at www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives (CC-BY-NC-ND) 4.0 license. Funded by Ministerie van Defensie Financieel Administratie en Beheerkantoor FABK Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Names: Schraagen, Jan Maarten, editor. Title: Responsible use of AI in military systems / edited by Jan Maarten Schraagen. Other titles: Responsible use of artificial intelligence in military systems Description: First edition. | Boca Raton : CRC Press, 2024. | Series: Chapman & hall/crc artificial intelligence and robotics series | Includes bibliographical references and index. Identifiers: LCCN 2023056193 (print) | LCCN 2023056194 (ebook) | ISBN 9781032524306 (hardback) | ISBN 9781032531168 (paperback) | ISBN 9781003410379 (ebook) Subjects: LCSH: Artificial intelligence—Military applications. | Artificial intelligence—Moral and ethical aspects. Classification: LCC UG479 .R47 2024 (print) | LCC UG479 (ebook) | DDC 355.40285/63—dc23/eng/20240229 LC record available at https://lccn.loc.gov/2023056193 LC ebook record available at https://lccn.loc.gov/2023056194 ISBN: 978-1-032-52430-6 (hbk) ISBN: 978-1-032-53116-8 (pbk) ISBN: 978-1-003-41037-9 (ebk) DOI: 10.1201/9781003410379 Typeset in Times by codeMantra
Page 6
v Contents Preface viii Acknowledgements ix Editor x Contributors xi 1 Introduction to Responsible Use of AI in Military Systems 1 Jan Maarten Schraagen SECTION I Implementing Military AI Responsibly: Models and Approaches 15 2 A Socio‑Technical Feedback Loop for Responsible Military AI Life‑Cycles from Governance to Operation 17 Marlijn Heijnen, Tjeerd Schoonderwoerd, Mark Neerincx, Jasper van der Waa, Leon Kester, Jurriaan van Diggelen, and Pieter Elands 3 How Can Responsible AI Be Implemented? 37 Wolfgang Koch and Florian Keisinger 4 A Qualitative Risk Evaluation Model for AI‑Enabled Military Systems 59 Ravi Panwar 5 Applying Responsible AI Principles into Military AI Products and Services: A Practical Approach 87 Michael Street and Sandro Bjelogrlic 6 Unreliable AIs for the Military 101 Guillaume Gadek
Page 7
vi Contents SECTION II Liability and Accountability of Individuals and States 125 7 Methods to Mitigate Risks Associated with the Use of AI in the Military Domain 127 Shannon Cooper, Damian Copeland, and Lauren Sanders 8 ‘Killer Pays’: State Liability for the Use of Autonomous Weapons Systems in the Battlespace 153 Diego Mauri 9 Military AI and Accountability of Individuals and States for War Crimes in the Ukraine 169 Dan Saxon 10 Scapegoats!: Assessing the Liability of Programmers and Designers for Autonomous Weapons Systems 192 Afonso Seixas Nunes, SJ SECTION III Human Control in Human–AI Military Teams 211 11 Rethinking ‘Meaningful Human Control’ 213 Linda Eggert 12 AlphaGo’s Move 37 and Its Implications for AI‑Supported Military Decision‑Making 232 Thomas W. Simpson 13 Bad, Mad, and Cooked: Moral Responsibility for Civilian Harms in Human–AI Military Teams 248 S. Kate Devitt 14 Neglect Tolerance as a Measure for Responsible Human Delegation 278 Christopher A. Miller and Richard G. Freedman SECTION IV Policy Aspects 301 15 Strategic Interactions: The Economic Complements of AI and the Political Context of War 303 Jon R. Lindsay
Page 8
Contents vii 16 Promoting Responsible State Behavior on the Use of AI in the Military Domain: Lessons Learned from Multilateral Security Negotiations on Digital Technologies 318 Kerstin Vignard SECTION V Bounded Autonomy 343 17 Bounded Autonomy 345 Jan Maarten Schraagen Index 371
Page 9
viii Preface The Netherlands organized an international summit (REAIM) on the topic of respon‑ sible AI in military systems on February 15‑16, 2023. Leading up to this summit, vari‑ ous activities were organized. On behalf of the Dutch Ministry of Defence, an Expert Workshop on the Responsible Use of AI in Military Systems was organized on October 31 and November 1, 2022. The venue was the Naval Establishment in Amsterdam. Thirty academic experts and twenty‑five representatives from government, industry, and research institutes came together for two days of highly interactive discussions on how to move the responsible development, deployment, and use of AI in the military domain forward. This Expert Workshop was followed up by an invitation to the aca‑ demic experts to write a chapter addressing the issues in their field of expertise regard‑ ing the responsible use of AI in military systems. The chapters were written in the first half of 2023, taking into account both the outcomes of the Expert Workshop and the REAIM Summit, as well as the latest developments in AI, such as the rise of ChatGTP, which was released in November 2022, just after the Expert Workshop. The resulting chapters in this book therefore represent the state of the art in the quickly developing field of Responsible AI. They are written from various perspectives and academic disciplines, including, law, ethics, computer science, human factors engi‑ neering, and policy making.
Page 10
ix Acknowledgements First and foremost, I would like to thank the Dutch Ministry of Defence for providing me with the opportunity to organize the Expert Workshop on the Responsible Use of AI in Military Systems and to contribute to the organization of the REAIM Summit. Without these activities, and without the Dutch MoD’s continued funding and support, this book would not have been possible. In particular, I would like to thank Auke Venema of the Dutch MoD for his unwavering belief in me. Auke was the one responsible for hiring me during my secondment to the Dutch MoD and he stimulated me to take up the many activities of which this book is but one of the results. He also guided me through the intricacies of the MoD’s bureaucracy and politics, making it much less of a maze than it otherwise would have been. Many thanks also to Geert Kuiper, Merle Zwiers, Josanne van Gorkum, Merel Roolvink, Lotte Kerkkamp (now with TNO), Joël Postma, Michiel van Dusseldorp, Tom van Hout, and Mirke Beckers, all, with the Dutch MoD. I have thoroughly enjoyed all of your support in the run‑up to the REAIM Summit. Thanks also to the team at the Ministry of Foreign Affairs with whom I collabo‑ rated for almost a year while organizing the REAIM Summit: Tessa de Haan, Maarten Wammes, Eline Bötger, Ingeborg Denissen, Veerle Sonneveld, Maud Duit, Maaike Stroeks, Thomas Kist, Pieter Ton, Robin Middel, and Maarit Wittebrood. This book would also not have been possible without the many stimulating discus‑ sions I had with my colleagues at TNO: Jurriaan van Diggelen, Antoine Smallegange, Adelbert Bronkhorst, Marlijn Heijnen, Liisa Janssens, Leon Kester, Pieter Elands, Jasper van der Waa, Tjeerd Schoonderwoerd, Mark Neerincx, and Freek Bomhof. In particu‑ lar, I would like to thank Jurriaan van Diggelen en Liisa Janssens for their detailed and constructive criticism of my concluding chapter. They cannot be held responsible for any remaining errors in the chapters I wrote for this book. I would like to especially sin‑ gle out Chris Jansen for his continued support in his capacity as my research manager. Jan Maarten Schraagen
Page 11
x Editor Jan Maarten Schraagen is a Principal Scientist at The Netherlands Organization for Applied Scientific Research (TNO). His research interests include human‑autonomy teaming and responsible AI. He is the main editor of Cognitive Task Analysis (2000) and Naturalistic Decision Making and Macrocognition (2008) and co‑editor of the Oxford Handzbook of Expertise (2020). He is editor‑in‑chief of the Journal of Cognitive Engineering and Decision Making. Dr. Schraagen holds a PhD in Cognitive Psychology from the University of Amsterdam, The Netherlands.
Page 12
xi Contributors Sandro Bjelogrlic Data Science & Artificial Intelligence Centre, Chief Technology Office NATO Communications and Information Agency The Netherlands Shannon Cooper Department of Defence Australia Defence Force Australia Damian Copeland Department of Defence University of Queensland Australia S. Kate Devitt Human‑centred computing, ITEE University of Queensland Australia Linda Eggert Institute for Ethics in AI, and Balliol College University of Oxford UK Pieter Elands Intelligent Imaging TNO The Netherlands Richard G. Freedman Smart Information Flow Technologies USA Guillaume Gadek Engineering – Connected Intelligence Airbus Defence and Space France Marlijn Heijnen Human Machine Teaming TNO The Netherlands Florian Keisinger Airbus Defence and Space Germany Leon Kester Autonomous Systems & Robotics TNO The Netherlands Wolfgang Koch Dept. Sensor Data & Information Fusion Fraunhofer FKIE/University of Bonn Germany Jon R. Lindsay School of Cybersecurity and Privacy, and the Sam Nunn School of International Affairs Georgia Institute of Technology USA Diego Mauri Department of Law University of Florence Italy
Page 13
xii Contributors Christopher A. Miller Smart Information Flow Technologies USA Mark Neerincx Human Machine Teaming TNO The Netherlands Ravi Panwar United Services Institution of India India Lauren Sanders School of Law University of Queensland Australia Dan Saxon Leiden University College The Netherlands Tjeerd Schoonderwoerd Human Machine Teaming TNO The Netherlands Afonso Seixas Nunes, SJ School of Law St. Louis University USA Thomas W. Simpson Blavatnik School of Government University of Oxford UK Michael Street Data Science & Artificial Intelligence Centre, Chief Technology Office NATO Communications and Information Agency The Netherlands Jasper van der Waa Human Machine Teaming TNO The Netherlands Jurriaan van Diggelen Human Machine Teaming TNO The Netherlands Kerstin Vignard National Security Analysis Department (APL) and Johns Hopkins Institute for Assured Autonomy Johns Hopkins Applied Physics Lab USA
Page 14
DOI: 10.1201/9781003410379-1 CC BY‑ND ‑ Attribution‑NoDerivs 1 1Introduction to Responsible Use of AI in Military Systems Jan Maarten Schraagen Technological developments in Artificial Intelligence (AI) continue to add new dimensions and complexities to world security and future conflict scenarios at an increasing pace. While the application of AI holds great potential for progress and economic growth as well as significant opportunities in the fields of security and defense, its potential misuse in international crises and conflicts may undermine the world’s security interests and cre‑ ate risks for international peace and stability. The international community is now faced with the central question of how military application of AI can – and should – be dealt with responsibly while at the same time creating an effective deterrent. This Introduction will set the stage for the chapters that follow, by providing a brief overview of relevant developments in AI, the military, as well as systems engineering practices. This will be followed by a brief introduction to each chapter, providing the reader with an overview of the contents of this volume. ARTIFICIAL INTELLIGENCE: A BRIEF HISTORY AI has a long and varied history, with periods of scientific and commercial successes followed by periods of disillusionment, instigated by scientific challenges as well as unrealistically high expectations (Nilsson, 2009). In the early days of AI (1956–1974),
Page 15
2 Responsible Use of AI in Military Systems the objective of making machines intelligent was primarily conceived as implementing general search strategies that could reason over symbolic task representations. However, it gradually became apparent that these general search strategies were insufficient for attaining high levels of performance. Researchers subsequently turned to ways of incor‑ porating large amounts of domain knowledge into systems. AI moved from a search paradigm to a knowledge‑based paradigm (Goldstein & Papert, 1977), culminating in the heyday of highly domain‑specific expert systems in the 1980s (Feigenbaum et al., 1988). However, expert systems were brittle, meaning they only performed well on the limited scope they were designed for, and with the assistance of human experts who were required to close the gap between the designers’ intentions and the real‑world application (Woods, 2016). In a particular study on fault diagnosis with an expert system, technicians were required to follow underspecified instructions by the expert system, to infer machine intentions, and to recover from errors that led the expert system off‑track (Roth et al., 1987). It should come as no surprise that expert systems did not live up to their expectations and rarely made it out of the lab to real‑life usage (Leith, 2016). For a long time (roughly from 1990 until 2010), several alternative approaches (e.g., multiagent systems and the Semantic Web) were explored, with little to no success. Then, big data and machine learning entered the scene (Russell & Norvig, 2021). Deep learning turned out to be very successful, leading to unprecedented outcomes such as superhuman performance on image classification tasks, game‑playing (Go, chess), and major breakthroughs in voice recognition and automatic language translation. Deep Neural Networks (DNNs) seem to bypass the problem of manual knowledge elicita‑ tion and modeling common‑sense knowledge that haunted expert systems in the 1980s. However, manual labeling work is still required, for deep learning image classifiers still require labels in order to be able to learn. To obtain a label (for instance, that a certain image qualifies as a ‘cat’ and another as a ‘dog’), a dataset usually requires humans to point out the area and indicate which type of object resides there. As deep learning requires a lot of data, this burden of manual labeling work is often too large or simply not feasible. A second problem with DNNs is that they are no longer understandable by humans. Performing calculations with tens of millions of parameters, the function‑ ing of a deep learning network is inherently incomprehensible to humans (the problem of so‑called ‘black‑box AI models’). Finally, DNNs may turn out to be brittle after all, as small perturbations in the input image may easily fool a neural image classifier (Moosavi‑Dezfooli et al., 2016). In conclusion, AI is still very much in development and a future AI era may well go beyond deep learning and evolve into a hybrid of multiple connectionist AI techniques, symbolic approaches, and humans handling unexpected situations that inevitably arise (Peeters et al., 2021). ARTIFICIAL INTELLIGENCE IN MILITARY SYSTEMS In the past, AI was funded largely by defense‑related funds. This changed around 2010 when AI became a huge commercial success, giving rise to billion‑dollar civilian
Page 16
1 • Introduction to Responsible Use of AI in Military Systems 3 industries in highly automated driving and data analytics. Still, the recent developments in AI have not gone unnoticed by the defense sector. AI is generally viewed as having large promises in a number of defense areas. AI is expected to speed up and improve decision‑making processes, as it is able to process large amounts of data at speeds that are not matched by humans. AI may also be able to select the right information out of large amounts of data, thereby enhancing decision‑making processes. AI may also be used to control robots and information agents that can perform dull, dirty, and danger‑ ous tasks without a human operator, thereby freeing up already scarce personnel to focus on more demanding cognitive tasks. Instead of a single tele‑operated robot, such as a drone, AI may be used not only to free up personnel, but also to scale up to numer‑ ous drones. Also, in communication‑denied environments (e.g., underwater or through jamming), where tele‑operation is impossible, AI can enable autonomy. The applica‑ tions of AI lie in several military domains, such as unmanned autonomous systems, decision‑making support and intelligence, cyber security, logistics and maintenance, business processes (HR, training, medical, automating work processes), and safety (own personnel as well as civilians). AI can enhance power on the battlefield, as well as efficiency and effectiveness in the use of unmanned autonomous systems. It can also make work more attractive by delegating particular dull, dangerous, and dirty tasks to AI. If AI takes over certain dan‑ gerous tasks, it may make the work of military personnel safer. In military decision sup‑ port and intelligence, AI can perform automated analysis, combination, and selection of huge amounts of data. This may enhance situation awareness and sensemaking on the battlefield, as well as speed up and qualitatively improve the intelligence‑gathering process. AI may also play a role in the automated detection of attacks and vulnerabili‑ ties. AI may do this orders of magnitude faster than humans. Also, AI may assist in the automated analysis of the condition of systems, enabling better and faster predictive maintenance and proactive logistics. AI may assist in the automation of work processes, recruitment of personnel, training and education of personnel, as well as in health moni‑ toring and diagnosis. In conclusion, there are many potential applications of AI in military systems, going beyond merely weapons systems. It is also important to stress that AI will be used to enhance current systems rather than act as a stand‑alone ‘AI system’. This implies that AI will be used as an add‑on to existing systems in the domains mentioned above. CHALLENGES OF USING AI IN THE MILITARY DOMAIN Apart from the perceived benefits, there are also challenges associated with the use of AI. First, if AI‑based solutions are to be used, they need to be trusted. This is achieved with sound development and validation methods at different phases of a system’s life cycle. This in turn requires explainability, so that the developers and certification authorities can scrutinize the solution. Explainability is defined here as the capabil‑ ity of an AI agent to “produce details or reasons to make its functioning clear or easy
Page 17
4 Responsible Use of AI in Military Systems to understand” (Arrieta et al., 2020, p. 85). Moreover, in some cases also the user or regulator could scrutinize the results of an AI‑based solution if it were explainable. As mentioned above, DNNs are no longer understandable by humans and currently have a hard time explaining themselves. The field of ‘Explainable AI’ is rapidly developing and has grown exponentially over the past few years (Arrieta et al., 2020). Hence, the challenges associated with this topic will remain with us for the foreseeable future. One particular research challenge, to be discussed in more detail below, is what trust repair strategies should be adopted by intelligent teammates working in human‑agent teams. Second, to the extent that large data sets are used by the AI, there is a risk that the data sets are biased. For example, they may work for white males but not for black females, thus leading to discrimination of particular groups in society. There is also the related risk that, as the world constantly changes, there will be ‘distributional drift’ or ‘prediction drift’ in the data. In settings with significant changes/distribution shifts, the model based on the past data may not survive contact with the world as it currently is (a state of affairs that has long been recognized in the military, as witnessed by the saying that ‘no plan survives first contact with the enemy’). Therefore, the model needs to be monitored and the data need to be as unbiased as possible. This is important not only from an ethical point of view, but also from a performance point of view (biased AI may simply not be effective in particular situations). On the other hand, to the extent that the military is bound by legal obligations on data gathering, as well as dealing with inherently complex situations with a lot of contextual factors, there may in many cases actually be a shortage of data, while the demand for data may be much higher than in civilian settings (e.g., in e‑commerce). This may also negatively impact the quality of the models developed in AI. Third, to the extent that AI takes over certain tasks from humans, there is a fear of humans not being in control anymore over what AI does. This plays a role in the dis‑ cussion on the use of AI in autonomous weapons systems (AWS). Given the difficulties associated with clearly defining ‘meaningful human control’, and the fact that ‘control’ is not a requirement, whereas compliance with the law of war is, the U.S. Department of Defense (2023) prefers the term ‘appropriate levels of human judgment’ instead of ‘meaningful human control’. In response to this, Human Rights Watch (2023) claims that it is not clear what constitutes an “appropriate level” of human judgment. Human Rights Watch also claims that human “control” is an appropriate word to use because it encompasses both the mental judgment and physical act needed to prevent AWS from posing moral, ethical, legal, and other threats. Hence, the debate on the use of the word ‘control’ is far from over. To make matters more complicated, Ekelhof (2019) has right‑ fully pointed out that control is distributed over multiple persons at various junctions in the decision‑making cycle involved in the target selection and engagement process. Therefore, different forms of control are exercised even before weapons are activated. And even after an AWS has been activated, there may be a human ‘in the loop’ or ‘on the loop’, leading to disengagement of the weapon system prior to impact (this is not the case for all AWS; moreover, this discussion largely depends on one’s definition of what an AWS is). This leads us, finally, to the issue of the definition of ‘autonomous weapons systems’ or ‘autonomy’ in particular. The arguments surrounding this definition are highly contested as well. The UN Convention on Certain Weapons (CCW) established
Page 18
1 • Introduction to Responsible Use of AI in Military Systems 5 a Governmental Group of Experts (GGE) to discuss emerging technologies in the area of lethal autonomous weapon systems (LAWS). Over the period 2014–2019, the CCW/ GGE has not arrived at a shared definition of AWS. Indeed, in a recent review, Taddeo and Blanchard (2022) identified 12 definitions of AWS proposed by States or key inter‑ national actors. Clearly, this approach is detrimental in facilitating agreement around conditions of deployment and regulation of their use. However, for the purposes of this article, the discussions surrounding LAWS should not be confused with discussions on the use of AI in military systems. Autonomy in military systems may be enabled by AI, but there are also other technologies to enable autonomy. RESPONSIBLE USE OF AI While automation based on AI holds great potential for the military domain, it can also have unintended adverse effects due to various imperfections introduced throughout the life cycle. This can be due to biased data, wrong modeling assumptions, etc. In order to advance the trustworthiness of AI‑enabled systems, and hence their ultimate use, an iterative approach to the design, development, deployment, and use of AI in military systems is required. This approach, when incorporating ethical principles such as lawfulness, traceability, reliability, and bias mitigation, is called ‘Responsible AI’ (U.S. Department of Defense, 2022). This implies that the military use of AI will be conducted in a recognized, responsible fashion across the enterprise, mission support, and operational levels in accordance with international law. The normative statements below constitute a first step toward the responsible use of AI in military systems. It is important to recognize that Responsible AI is not identical to ‘explainability’ or ‘transparency’, and therefore should not be confused with the field of Explainable AI. An AI model is considered to be transparent if by itself it is understandable (Arrieta et al., 2020), hence without the need for further explanations. Responsible AI involves other ethical principles besides explainability or transparency, such as lawfulness, bias mitigation, and reliability. In that sense, it encompasses explainable AI but cannot be reduced to it. In terms of incorporating ethical principles such as data protection and bias miti‑ gation, safe and secure AI will be enabled by the development of sustainable, privacy‑ protective data access frameworks that foster better training and validation of AI models utilizing quality data. Proactive steps should be taken to minimize any unintended bias in the development and use of AI applications. Adequate data protection frameworks and governance mechanisms should be established first within Defense and next with industry at the national or international level, protected by judicial systems, and ensured throughout the life cycle of AI systems. AI applications should be appropriately understandable and transparent, includ‑ ing through the use of review methodologies, sources, and procedures. To this end, AI applications should provide meaningful information, appropriate to the context and user, and consistent with the state of art. Transparency and explainability are factors
Page 19
6 Responsible Use of AI in Military Systems that can improve human trust in AI systems. The level of transparency and explainabil‑ ity should always be appropriate to the context and impact, as there may be a need to balance between transparency and explainability and other principles such as privacy, safety, and security. An iterative socio‑technical systems engineering and risk management approach should be adopted to ensure potential Al risks (including privacy, digital security, safety, and bias) are considered from the outset of an Al project. Efforts should be taken to mitigate or ameliorate such risks and reduce the likelihood of unintended consequences. A robust testing process should be developed, allowing for the assessment of AI applica‑ tions in explicit, well‑defined use cases. This includes continuous identification, evalu‑ ation, and mitigation of risks across the entire product lifecycle and well beyond initial deployment. Appropriate oversight, impact assessment, audit, and due diligence mechanisms should be developed to ensure accountability for AI systems and their impact through‑ out their life cycle. Both technical and institutional designs should ensure auditability and traceability of (the working of) AI, in particular to address any conflicts with human rights norms and standards and threats to environmental and ecosystem well‑being. AI actors should ensure traceability, including in relation to datasets, processes, and decisions made during the AI system lifecycle, to enable analysis of the AI system’s outcomes and responses to inquiry, appropriate to the context and consistent with the state of art. CONCEPTUAL DISTINCTIONS AND CLARIFICATIONS It is important to make a number of conceptual distinctions and clarifications, particu‑ larly when talking about the responsible use of AI in military systems. First, in response to recent fast developments in AI, many organizations, agencies, and companies have published AI ethics principles and guidelines. In a meta‑analysis, Jobin, Ienca, and Vayena (2019) included 84 documents containing ethics principles and guidelines. The most frequently mentioned principles were: transparency, justice and fairness, non‑maleficence, responsibility, and privacy. The principles and guidelines have been criticized by some for being (i) too abstract to be practical, (ii) reflecting mainly the values of the experts chosen to create them (hence, not being inclusive), and (iii) serving the priorities of the private entities which funded some of this work (‘ethics washing’) (Hagendorff, 2020; Hickok, 2021). Although some of these criticisms are justified, one should realize that the principles are a starting point. There is great value in all of these documents being publicly accessible (several websites track them and make them available for analysis purposes, e.g., aiethicslab.com and algorithmwatch. org). Some of these principles are useful for structuring the discussion regarding the challenges for human use, for instance, bias mitigation, explainability, traceability, gov‑ ernability, and reliability (taken from the North Atlantic Treaty Organization (NATO) Principles of Responsible Use of AI, 2021).
Page 20
1 • Introduction to Responsible Use of AI in Military Systems 7 Second, ‘military systems’ are much broader than just weapons systems. AI may be of use in a broad array of systems and applications, including business process applica‑ tions, predictive maintenance, and highly automated responses to cyber‑attacks. This does not in any way diminish the importance of discussing the use of AI in (offensive) weapons systems. Third, ‘autonomy’ and ‘AI’ are not identical. AI may be used to achieve the goal of system autonomy, in the general (and, admittedly, vague) sense of achieving tasks with little or no human intervention (Endsley, 2017). However, there are other ways of achieving this goal, including the use of logic‑based programming as used in clas‑ sical automation. An example of the latter would be close‑in weapon systems, such as the Goalkeeper or the Phalanx, which are completely automatic weapon systems for short‑range defense of ships. These weapon systems may be called ‘autonomous’ as defined in the U.S. DoD Directive 3000.09 (2023): “A weapon system that, once acti‑ vated, can select and engage targets without further intervention by an operator”. Yet, these close‑in AWS do not need AI to function as intended. This is not to deny that data and AI may be key enablers of autonomy. Fourth, the definition of the concept of ‘autonomy’ is driven by political and stra‑ tegic motivations, as briefly discussed above, and is not value‑neutral. It is beyond the scope of the current chapter to arrive at a value‑neutral definition of ‘autonomy’ (see Taddeo and Blanchard, 2022, for such an attempt). I will take up the issue of how to define autonomy in the final concluding chapter of this volume. OVERVIEW OF THE CHAPTERS IN THIS BOOK The chapters in this book are organized into four major sections. Section I presents models and approaches for implementing military AI responsibly. Section II is an over‑ view of legal aspects regarding the liability and accountability of individuals and states when using AI in the military domain. Section III addresses the shifting role of human control in military teams in which humans and AI have to work together. This section includes both philosophical and human factors contributions. Section IV broadens the scope to include political and economic aspects of using AI in the military domain. Section V contains a concluding chapter in which the issues addressed in the previous sections are critically evaluated. Below, I will briefly summarize the contents of each chapter. Section I: Implementing Military AI Responsibly: Models and Approaches This section starts with the chapter by Heijnen et  al. who present a Socio‑Technical Feedback loop (SOTEF) methodology to establish and maintain the required value alignment at the levels of governance, design, development, and operation of military AI throughout its life cycle. Value alignment is important as the use of military AI
The above is a preview of the first 20 pages. Register to read the complete e-book.

Recommended for You

Loading recommended books...
Failed to load, please try again later

Tip the Site

Scan the WeChat Pay or Alipay code to tip. No login required.

WeChat Pay
Alipay
← Back to List