Military Review English Edition May-June 2014 | Page 15

ROBOTIC WARFARE In principle, humans retain control of—and responsibility for—the behavior of autonomous machines. However, establishing precisely who is responsible for a machine’s behavior is challenging. Autonomous machines, no matter how they are defined, developed, or used, operate as part of broad sociotechnical systems involving numerous individuals and organizations. We advocate the concurrent development of new responsibility practices with the development of new technologies rather than before or after those technologies are developed and adopted for use. This is necessary because literature in the field of science and technology studies shows that the trajectory of a te chnology’s development is unpredictable; how a technology takes shape depends on complex negotiations among relevant social groups.4 The technologies eventually adopted and used are not predetermined by nature or any other factor. No one can predict with certainty how a developing technology will turn out or what new technologies will emerge. In the course of development, a new technology may change in response to many factors, including changes in funding, historical events such as wars, changes in the regulatory environment, and market indicators. The technologies that succeed (i.e., that are adopted and used) are the outcome of complex negotiations among many actors, including engineers and scientists, users, manufacturers, the public, policymakers, politicians, and others. Negotiations among the actors involved with a new technology are part of the overall discourse around that technology from its earliest stages of development. The discourse about responsibility and autonomous military robots is a case in point; current discourse provides an opportunity to observe issues of responsibility being worked out early in the technology’s development. The negotiations between researchers, developers, engineers, philosophers, policymakers, military authorities, lawyers, journalists, and human rights activists are taking place in the media and academic journals, at conferences and trade shows, through drafting new policies and regulations, in negotiating international treaties, and also through designing and developing the technologies. This process contrasts starkly with the all-too-common idea that issues of responsibility are decided after MILITARY REVIEW May-June 2014 a technology is developed or separately from technological design. Framing robots as autonomous challenges ordinary notions of responsibility. Autonomy in daily life and moral philosophy implies acting on one’s own, controlling one’s self, and being responsible for one’s actions. On the other hand, being responsible generally means that individuals have some kind of influence or control over their actions and the outcomes of those actions. The idea of the autonomy of robots suggests that humans are not in control of the robots. Hence, at first glance, it may seem that humans should not be held responsible for autonomous robot behavior. However, this narrative of future autonomous robots operating on their own, without human control, is somewhat misleading, and it draws attention away from important choices about responsibility—choices made at the level of design and implementation. Our analysis of the discourse on autonomous artificial agents and responsibility shows that delegating tasks to autonomous technologies is compatible with holding humans responsible for the behavior of those technologies. This is so for at least two reasons. First, the definition of machine autonomy has numerous interpretations, but all involve various kinds and degrees of human control. Second, humans decide who is responsible for the actions of a machine. Their decisions are affected by, but not entirely determined by, the nature of technology. Responsibility for the behavior of autonomous machines is and must continue to be determined by ongoing negotiations between relevant interest groups during the development of new technologies. Negotiating Autonomy Popular accounts of future military robots often portray these technologies as entities with capabilities that rival or surpass those of humans. We are told that robots of the future will have the ability to think, perceive, and even make moral decisions. In Discover Magazine, for instance, Mark Anderson writes, “As surely as every modern jetliner runs primarily on autopilot, tomorrow’s military robots will increasingly operate on their own initiative. Before the decade is out, some fighting force may well succeed in fielding a military robot that can kill without a joystick operator behind a curtain 13