The debate over autonomous weapons systems.

• Author: Noone, Gregory P.
• Position: Frederick K. Cox International Law Center Symposium: International Regulation of Emerging Military Technologies

The debate over Autonomous Weapon Systems (AWS) has begun in earnest with advocates for the absolute and immediate banning of AWS development, production, and use arguing AWS should be banned because these systems lack human qualities, such as the ability to relate to other humans and to apply human judgment, that are necessary to comply with the law. In addition, the weapons would not be constrained by the capacity for compassion, which can provide a key check on the killing of civilians. The opposing viewpoint in this debate articulates numerous arguments that generally include: it is far too premature and too speculative to make such a proposal/demand; the Law of Armed Conflict should not be underestimated in its ability to control AWS development and future operations; A WS has the potential to ultimately save human lives (both civilian and military) in armed conflicts; A WS is as inevitable as any other technology that could potentially make our lives better; and to pass on the opportunity to develop AWS is irresponsible from a national security perspective. The purpose of this article is to help refine the AWS debate.

CONTENTS I. INTRODUCTION II. WHAT ARE AUTONOMOUS WEAPONS SYSTEMS? III. COMMON GROUND A. LAW OF ARMED CONFLICT B. HUMAN ERROR C. MACHINES INSTEAD OF HUMANS IV. TECHNOLOGY V. CONCLUSION I. INTRODUCTION

The debate over Autonomous Weapon Systems (AWS) has begun in earnest with advocates for the absolute and immediate banning of AWS development, production, and use planting their flag first. They argue that AWS should be banned because these systems lack human qualities, such as the ability to relate to other humans and to apply human judgment, that are necessary to comply with the law. In addition, the weapons would not be constrained by the capacity for compassion, which can provide a key check on the killing of civilians. (2) The opposing viewpoint in this debate articulates numerous arguments that generally include: it is far too premature and too speculative to make such a proposal/demand; the Law of Armed Conflict should not be underestimated in its ability to control AWS development and future operations; AWS has the potential to ultimately save human lives (both civilian and military) in armed conflicts; AWS is as inevitable as any other technology that could potentially make our lives better; and to pass on the opportunity to develop AWS is irresponsible from a national security perspective. (3) Some of the most respected and brilliant lawyers in this field are on opposite sides of this argument. The purpose of this article is to help refine the AWS debate.

2. WHAT ARE AUTONOMOUS WEAPONS SYSTEMS?

   The International Committee of the Red Cross defines Autonomous Weapon Systems as weapons that can "independently select and attack targets, i.e. with autonomy in the 'critical functions' of acquiring, tracking, selecting and attacking targets." (4) The U.S. Department of Defense defines AWS as: "a weapon system that, once activated, can select and engage targets without further intervention by a human operator. This includes human-supervised autonomous weapon systems that are designed to allow human operators to override operation of the weapon system, but can select and engage targets without further human input after activation." (5)

   AWS is not artificial intelligence. (6) There will not be "human qualities such as consciousness, emotion, sociability, semantic understanding required for human moral decision making." (7) AWS also isn't a Terminator science fiction movie scenario. (8) "SkyNet" is not going to take over the world. (9) Unless of course you think we are from the future and we're here to convince you autonomous weapon systems should be developed and trust us there is no Terminator scenario. In that case, have Sarah Connor give us a call. (10) "The autonomous robots being discussed for military applications are closer in operation to your washing machine than to a science fiction Terminator. The way the term 'autonomy' is used in robotics should not be confused with how the term is used in philosophy, politics, individual freedom or in common parlance. It is more related to the term automatic. An automatic robot carries out a pre-programmed sequence of operations or moves in a structured environment. A good example is a robot painting a car." (11)

   Autonomous Weapon Systems would be able to select and engage targets without human involvement in an unstructured environment. This is really the crux of the argument. Should weapons be developed that do not have a human in the "loop" (i.e. a closed "loop" system)? There are three types of weapon systems and they are generally described as:

   1. Human-in-the-loop or semi-autonomous systems require a human to direct the system to select a target and attack it, such as Predator or Reaper UAVs.

   2. Human-on-the loop or human-supervised autonomous systems are weapon systems that select targets and attack them, albeit with human operator oversight; examples include Israel's Iron Dome and the U.S. Navy's Phalanx Close In Weapons System (or CIWS).

   3. Human-out-of-the-loop or fully autonomous weapon systems can attack without any human interaction; there are currently no such weapons. (12)

   Similarly the U.S. Navy characterizes autonomous weapons in terms of mission complexity. "Supervised" weapons have human operators making the decisions (i.e. "human-in-the-loop" such as UAVs), "scripted" weapons carry out a "pre-planned script of the 'point, fire and forget' variety" (e.g. CIWS), and "intelligent" systems that are fully autonomous. (13) The U.S. Department of Defense has directed that "[a]utonomous and semi-autonomous weapon systems shall be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force."...