The 3 Laws of Robotics

Related Papers: AI Armageddon and the Three Laws of Robotics

As a high school student, I started reading some of Isaac Asimov’s short stories and novels.  One of the most prolific Science Fiction writers ever, Asimov credits himself as being the person to use the term “robotics” in print.  As an adult, I still enjoy his books and decided to take a new look at those dealing with a professional passion of mine: robots.  Of course, the thing that Asimov’s robots are most known for is their unfailing adherence to “The Three Laws of Robotics” introduced in his 1942 story, “Runaround”:

  1. A robot may not harm a human being, or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
  3. A robot must protect its own existence, as long as such protection does not conflict with the First or Second Law.

Without fail, if Asimov was writing about robots, then the story line included, and often centered around, the limitations of these laws under extreme circumstances.  While his stories highlighted the imperfections of the Laws, they also highlighted how unusual it was that a robot might even seem to have disobeyed the Laws for any reason.  The impression that comes out of all this is that the Three Laws are nearly perfect and only under the most improbable situations could a robot fail. 

In reading some of Asimov’s essays about his thoughts on robotics, I came to discover that Asimov fully believed that his Laws would be implemented in the robots of the future.  Not because of some egotistical belief in his own importance, but because he saw the laws as just an explicit formulation of the safety measures used in any human tool.  I began to wonder if he might not have a point and whether other researchers in the fields of AI and Robotics had every really considered the Three Laws from a serious implementation perspective.  To find out, I sent a handful of e-mails to AI and Robotics researchers that I knew of from papers or had met previously. To their credit, almost all of them replied and all with very interesting responses.  Below is the e-mail that I sent out, more or less.  It was personalized, of course, for those that I had personally met, but the individual aspects have been removed from both my e-mails and theirs.  Here was my initial e-mail:

    I am a researcher at the FedEx Institute of Technology at the University of Memphis currently working in the area of robotics. As I’m sure you are aware, there seems to be a general fear of intelligent robots from the public perspective fueled by the media and, more acutely, Hollywood.  Isaac Asimov, on the other hand, formulated the “Three Laws of Robotics” that, again, I am sure you are aware of but will restate them for clarification:

    1. A robot may not harm a human being, or, through inaction, allow a human being to come to harm.

    2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

    3. A robot must protect its own existence, as long as such protection does not conflict with the First or Second Law.

    These were meant to dissuade human fears and keep robots in check so to speak. Asimov viewed these not as verbal rules, but as mathematically/systematically embedded into a robot’s brain at such a low level that it could not break them.  I am trying to gauge the opinions of robotics and AI researchers regarding how implementable these laws might be in future intelligent robots.  For a more in-depth discussion of the laws, wikipedia has a pretty good article on it: http://en.wikipedia.org/wiki/Three_Laws_Of_Robotics

    I know your time is valuable and I greatly appreciate whatever you can spare to giving your opinion on this matter.

Here are the response I recieved in no particular order. All of the authors have given their permission for me to publish their emails.

David Bourne, a Principal Scientist of Robotics at Carnegie Mellon wrote:

    Well it's a good question!!

    The primary issue of robotics at this moment is whether the robot understands the semantics/pragmatics of its action. A robot certainly can follow its instructions, just the way a computer follows its instructions. But, is a given instruction going to crash a program or drive a robot through a human being? In the absolute, this answer is unknowable! But that is not the point. It is reasonable for the robot to try and determine the consequences of an action by some independent means. Is a human in the way of my robot arm during a rapid move? I'll look before I act. Of course, as I already stated, the looking may fail to see the danger. But look it should. Notice that even the looking is an action and it may in turn have additional consequences in a never ending spiral of self doubt/checking. That's why it is unknowable in the limit.

    Also, I would like to point out that the Third law is not a very good one. At this point in time humans are very error prone when it comes to programming computers and/or robots. Therefore, we consider it reasonable for the computer/robot to double check to make sure that it will not harm itself by the action. For example, the simple act of deleting a file is met with all sorts of objections and safe guards. This is particularly true, when you command a robot to travel faster or generate greater torques on its part than the robot design allows. Most robots prohibit this -- as they should!!

    Of course, Asimov assumes that pretty much all of the actors (humans and robots) understand the consequences of their actions. Furthermore, the third law is meant to represent a component in the chain of command (e.g., military chain of command). The private is supposed to go to their death if commanded by an officer. This assumes that the officer has superior knowledge about all aspects of the situation. Even this chain of command allows exceptions for field commanders who may have more detailed local knowledge. Everyone does their best, and if the rules are violated a military tribunal decides who made the right decision. All of these intricacies apply to robotics.

    We're still a long way off from a faithful application of Asimov's 3 laws of robotics.

    Sincerely, David
    Principal Scientist, Robotics
    Carnegie Mellon
    Pittsburgh, PA
     

Doug Blank, Associate Professor of Computer Science at Bryn Mawr College replied:

    The question about the three laws is an interesting point where fiction and science fiction are very far away from each other. The trouble is that robots don't have clear-cut symbols and rules like those that must be imagined necessary in the scifi world. Most robots don't have the ability to look at a person and see them as a person (a "human"). And that is the easiest concept needed in order to follow the rules. Now, imagine that they must also be able to recognize and understand "harm", "intentions", "other", "self", "self-preservation", etc, etc, etc. Then comes the hard part. Now they must be able to counterfactualize about all of those concepts, and decide for themselves if an action would break the rule or not. They would need to have a a very good idea of what will happen when they make a particular action.

    Even if a robot could do all of the above, then there is a big question of levels at which the rules must apply. For example, if you had a robot that could do the above, then it probably isn't following a set of easy-to-understand "rules", but its behavior is a much more complex emergent phenomena resulting from many lower-level systems. They would follow spoken rules about as well as humans follow spoken rules. On the other hand, if you tried to put the rules into the hardware, you don't know the language of the internal system, and couldn't hardwire them to do what you want (just like we can't do brain surgery to make people not break the law---how many millions of neurons would you need to adjust?)

    -Doug

Maja Mataric at USC wrote:

    You are correct that all roboticists are aware of Asimov's laws of robotics, but it not something that is taken seriously enough to even be included in any robotics textbooks, which tells you something about its role in the field. The general ideas are very good, but the basic notions of safety are not so simple as to be encoded in succint and independent rules that can potentially interfere with each other as Asimov cleverly described in many of his stories. In practice, robot control software that has to do with issues of safety is complex, distributed, and not so easily broken down into a hierarchy of rules.

    However, the overall goals of Asimov's rules are of course a priority of any robot system that interact with people, so the spirit is very much alive and well.

    Hope this helps.

    Cheers,

    <>Maja

    Maja J Mataric'

    Professor of Computer Science and Neuroscience Founding Director, USC Center for Robotics and Embedded Systems(cres.usc.edu) President-Elect, Academic Senate Director, USC Robotics Research Lab (robotics.usc.edu) Computer Science Department and Neuroscience Program University of Southern California

James Kuffner, Assistant Professor at The Robotics Institute of CMU stated:

    Thanks for your email. It is certainly an interesting topic of discussion. The problem with these laws is that they use abstract and ambiguous concepts that are difficult to implement as a piece of software. What does it mean to "come to harm"? How do I encode that in a digital computer? Ultimately, computers today deal only with logical or numerical problems and results, so unless these abstract concepts can be encoded under those terms, it will continue to be difficult. I have lots of other thoughts on the matter, but this is just a short summary.

    Best Regards,
    -James

    --
    James Kuffner
    Assistant Professor
    The Robotics Institute
    Carnegie Mellon University

     

Aaron Sloman, Professor of Artificial Intelligence and Cognitive Science at The University of Birmingham put up a web site about the Three Laws shortly after our exchange.  In his email to me, he wrote:

    > As I'm sure you are aware, there seems to be a general fear of
    > intelligent robots from the public perspective fueled by the
    > media and, more acutely, Hollywood.

    Whenever journalists etc. ask me about that I answer something like:

      It is very unlikely that intelligent machines could possibly produce more dreadful behaviour towards humans than humans already produce towards each other, all round the world even in the supposedly most civilised and advanced countries, both at individual levels and at social or national levels.

      Moreover, the more intelligent the machines are the less likely they are to produce all the dreadful behaviours motivated by religious intolerance, nationalism, racialism, greed, and sadistic enjoyment of the suffering of others. They will have far better goals to pursue.

    > Isaac Asimov, on the other hand, formulated the "Three Laws of
    > Robotics" that, again, I am sure you are aware of but will
    > restate them for clarification:...

    I have always thought these are pretty silly: they just express a form of racialism or speciesism.

    If the robot is as intelligent as you or I, has been around as long as you or I, has as many friends and dependents as you or I (whether humans, robots, intelligent aliens from another planet, or whatever), then there is no reason at all why it should be subject to any ethical laws that are different from what should constrain you or me.

    My 1978 book, The computer revolution in philosophy had an epilogue on cruelty to robots:

       http://www.cs.bham.ac.uk/research/cogaff/crp/

    > These were meant to dissuade human fears and keep robots in check
    > so to speak. Asimov viewed these not verbal rules, but as
    > mathematically/systematically embedded into a robot's brain at
    > such a low level that it could not break them. I am trying to
    > gauge the opinions of robotics and AI researchers regarding how
    > implementable these laws might be in future intelligent robots.

    There is a prior question as to whether they *should* be implemented, as indicated above. I would regard that as unethical in some cases.

    The main obstacle to implementation is vagueness in the laws. E.g. what counts as 'harm'. There are many things that are regarded as good parental behaviour towards their children that might be regarded as harm if done by a stranger (e.g. fiercely telling off, or forcibly restraining, a child who has started to do something potentially very dangerous to him/herself).

    Another obstacle involves potential contradictions as the old utilitarian philosophers found centuries ago: what harms one may benefit another, etc., and preventing harm to one individual can cause harm to another. There are also conflicts between short term and long term harm and benefit for the same individual. There is nothing in Asimov's formulation about how the contradictions should be resolved, though I know he and others have noticed the problems and explored some options.

    (You can look up the history of 'utilitarianism' as an ethical theory if you want to know more. The main exponent was John Stuart Mill, but many have challenged or tried to extend his theories.)

    But my main answer remains: humans are among the things on earth to be most feared. If we can find ways to educate and socialise them so that we no longer produce human monsters, or monster cultures, then the same methods should apply to machines with huma-like intelligence.

    If the robots are not human like, then the responsibilities of the designers are no different from the responsiblities of other designers of complex systems that are potentially harmful, e.g. nuclear power plants, dangerous chemical plants, giant dams that disrupt ecosystems, weapons that can be put to evil uses, or which can accidentally cause disasters, etc. In comparison with those, the majority of threats from robots in the foreseeable future will be tiny, partly because AI progress is so hard (for reasons I have been writing about!)

    I hope this is of some use to you.

    Best wishes
    Aaron

I am very greatful to all of those that took the time to reply, a couple of whom I have had ongoing conversations with regarding this subject as well as topics that were raised in the resulting paper.  I welcome further comments on this page or the paper as long as I can post them on this page :-)

 

[Home] [Classes] [Links] [Research] [Papers] [javam] [Contact] [Events] [Personal]