Wendell Wallach. Artificial Morality.
Questions:
1. Wallach argues about a future in which computers have comprehensive rationality; that is, they do not suffer from the bounded rationality characteristic true of today’s humans and robots. I would argue forcefully that this is a seriously flawed argument, and that embodied robots will have a significant bounded rationality problem. First, agree or disagree with me and explain why in concrete terms. Second, if I am in fact right that this is a flawed perspective, explain the impact of this changed assumption on the key points of this article regarding artificial moral intelligence.
Wendell Wallach. Implementing Moral Decision Making Faculties in Computers and Robots.
Consider a home care robot for an elderly person. It helps with simple chores, like cleaning the floor, but also keeps its eye on the occupant. In cases where the person is having trouble- say, spending too long in the bathroom, it has the ability and option to reach out to the occupant’s children by contacting them. It has to weigh privacy concerns with maintenance of safety and health.
2. The top-down strategy that Wallach describes on pages 466-467- in computer science and AI terms, what are the key breakthroughs we would need in order to implement such top-down moral reasoning for our home care robot? How many years away do you think this is? – explain your estimate.
3. The bottom-up strategy that Wallach describes on page 467- restate just how is this robot becoming a moral reasoning system? Wallach suggests that this bottom-up approach has promise due to its embedded learning nature, so that morality is learned in a context of existence and action relevant to one’s own personal experience. Yet at the bottom of p.467 Wallach suggests porting the resulting, learned system from one robot to another (in effect, a cloning of moral reasoning). What technical challenges do you foresee in this bottom-up approach? How many years away do you think this may be?
Ethics and Robotics discussion 
mklingen said:
1. Robots of the future will certainly be bounded, and in more ways than humans. When he talks about the “seven items” experiment, he is making a category mistake in thinking that the “seven things” the human is thinking of are of the same kind as the millions of “things” a computer allegedly can hold in its “memory.” A human is in fact not holding just seven bytes or whatever of information in a couple of neurons, but is using a vast, inefficient apparatus of human language and memory to associate these seven things with millions of neurons in the brain – all while processing and understanding the world around him or her, and understanding the social context of trying to “remember” those seven things to begin with. If asked, the human could write a whole novel on each of those seven things using the petabytes of information already stored in his or her memory. What the computer “remembers” is trivial in contrast in both its size and scope.
Further, he makes the mistake of characterizing moral decisions as computations, and our emotional judgments about them merely as “heuristics.” What if the emotional judgements themselves are the important part? What if they embody the “computation?”
2. Any strategy would require the robot to understand in the first place that the elderly occupant is spending “too long” in the bathroom. This necessitates understanding of “the bathroom” and its “occupant,” and what a socially acceptable amount of time is. Let’s ignore these (currently intractable) problems and focus on the moral reasoning aspects. If the robot is an act utilitarian, it will need to be able to compute *future consequences* of its actions, presumably using some kind of simulation. This problem is wildly intractable. If it is a rule utilitarian, it would have to somehow match “the situation” to a particular rule it has about such situations. This is slightly more tractable, but its dubious as to whether “situations” can be categorized compactly as well as their representational rules. If it’s a deontologist, it either needs to do the same thing as the rule utilitarian, or do another calculation where it deduces from first logical principles and a set of rules what the right action should be. The only way this would be tractable is if the situation were distilled into a few simple symbols. This is pretty dumb as well. I just don’t see any top-down calculation which would allow the robot to make a good choice other than hard-coded behaviors specifically designed for by the programmers – and I wouldn’t describe that as “artificial morality”. So I would say this will never be possible.
3. In this scenario, the robot becomes a moral agent by interacting with the world and learning in much the same way a human does what is right and wrong. It learns simple heuristics based on an underlying emotional system with some built in rules + learning. This sounds better in principle, because we don’t have to imagine programming the rules of ethics ourselves, but I think its just an appeal to a homunculus. I think Wallach (and other proponents of such approaches) are imagining a little person inside the robot, growing. Since people are moral agents, a robot controlled by a little person would also be a moral agent – but we don’t ask exactly how this “learning” (in the real, human sense) can be accomplished. I think it presupposes solving *all* other problems in robotics simultaneously. Because to be a proper moral agent, a robot needs to be able to experience comparable things to that a proper growing human experiences – and in order to do that, it needs comparable capabilities to a human. I think anyone would agree this is ridiculous to think about at this time. It won’t happen.
joelsimon6 said:
1)
I believe Wallach’s argument main flaw is reducing rationality to an information problem.
“It is important to recognize that much of our use of values in decision making function as a compensation or short-cut method for handling the vast bulk of information that impinges on a decision.”
He uses deep blue as an example of massive processing power prevailing over human intelligence. Essentially brute forcing millions of chess moves is a very bad example to generalize to all human thought. Chess is a very specific case where the goal is clear and moves can be objectively valued. Chess is also a very closed system in that the language of its input is fixed. If the rules of chess were tweaked to add or remove pieces, Kasperov would be able to adjust his knowledge appropriately while Deep Blue would be at a loss demonstrating how still extremely bounded its ‘rationality’ is. I think it is too easy to attempt to reduce human rationality to a computational problem when it is fundamentally not. I believe that until a ‘robot’ can really feel an emotion it will never be able to truly understand them. Much of the article is on the premise that human functionality is replicable by a turing machine. For instance assuming that machines are approaching human emotional intelligence because they can begin to detect certain emotions. Faking human intelligence is never a full replica for actual human intelligence.
2 a)
Well, this is certainly a loaded question. In short, if ‘implements’ means do as well as a human then I would say there are no signs that show it will ever be possible. utilitarianism, for example, is not a series of hard laws that can be coded in but rather a structure of thinking that assumes the user is intelligent enough to know when it should be broken. Knowing when to break rules is part of being an ethical human so if I had to specify a breakthrough I would say it was full human level AI which I have no ability to predict if its even possible let alone a date.
2 b)
The term learning here is very misleading. The robots are ‘rewarded for behavior that is morally praiseworthy’ which essentially means they are being manually told which actions are moral or not according to the person or body responsible for ‘training’ them. I see this as no different from hardcoding in what response should be taken to some finite number or inputs. Actual ethical learning in humans consists of trying action and judging there consequences by our own internal measures, something entirely different. Until the robots can judge the consequences on their own they really rant learning. So again for technical challenges I will have to say a full AI which I still have no ability to predict.
Seun Aremu said:
1.Wallach said it best himself, Humans augment their decision-making capabilities with experience, feel, intuition, and judgment-skills, all which take years and various unique experiences to obtain. In order for a computer to have the capability of comprehensive rationality, they would have to have the capability to utilize all of these tools plus holding worldly information that cross cultural lines. Also, I believe that for a computer to be capable of comprehensive rationality, it would have to defy Simon’s explanation of bounded rationality, meaning, the computer would have to be able to both make decisions based on efficiency, maximization of profits and a decision that simply satisfies the question at hand. With my stated understanding, I would have to say that I agree with your arguments. Wallach asks the question ” what contains will we build in….should those restraints be treated as hard rules or soft guidelines”. Based on Simon’s definition of the idea of bounded rationality, I believe that these guidelines will be self defined during implementation.
2. One would need a central processing unit that holds the core governing rule that all care robots must obey, for example ” safety of human prevails all other decisions”. It is almost as if one is implementing the Ten Commandments of healthcare into the robots logic. This logic pattern would have to be one that does a continues check. For every decision made, the robot must check to understand that it is not violating it’s “Core Law”. Initially, I would want to say the timing is dependent on processing speed, but I believe the time for this to happen would be when computers are capable of comprehensive rationality.
3.The main technical challenge I see is time. the time it takes for one robot to completely learn all the cases and scenarios it must make moral decision about. Then we address the challenge required to transfer that memory to another robot. In that instance, I feel the transfer of memory could be cleverly device to be as simple as one transferring the ISO image of one hard drive to another. In terms of time, I anticipate about 15 years. Assuming that the scenarios are uploaded to the robot at a pace twice as fast as it would take a new born to reach
Nico said:
1) I think this paper suffers from the terminology problems that we read about last class, although some of the papers that were cited in Wallach’s paper are much much worse (see http://www.cs.yale.edu/homes/scaz/papers/Gold-RAS-08.pdf )
I also think that it is dangerous to view emotions for a computational perspective. For me, emotion will never be a power problem. This isn’t something you can throw flops at. Nor can it be so easily stated that “Loss of emotional control might be understood as sensory output that exceeds the computational capacity of the mind/body.” In fact the entire premise of this paper rests on the idea that emotion is essentially a lack of data problem, in the same way that having the power to simulate brains means we actually can model its reactions.
Anything that could be called artificial morality will be different from a human moral agent. The mind is simply too dependent on the body, on the human-ness of the entire system, to work and be trusted as true human thought.
(see the first part of this ENORMOUS pdf. ‘Can Thought Go On Without a Body?’ http://www.investigatingtheterror.com/documents/files/Lyotard%20The%20Inhuman%20Reflections%20on%20Time.pdf
Whether a non-human morality would be better is essentially moot. A friendly dog might be a more moral being than you or I, if we are talking in terms of its intentions and effect on the world, but that doesn’t mean it should be making moral decisions for me. It is too removed from my context for me to truly believe that it can weigh the consequences of human actions on a human scale.
That said, I don’t see the problem in programming systems with roomba-like emulation of a moral system. I think Wallach’s argument for bounded morality and substitutes for emotions are strong. I still think using the word ‘morality’ is massively misleading and should not be used. It puts too much stress on the machine’s supposed agency. Really one should say that most programs already have morality built into them, although these are really artifacts of the moral leanings of their creators, in such instances as DRM, GNU licensing, key tracking, and security back doors.
2) a) “The convergence of AI, genomics, and nanotechnology will give rise to future
technological possibilities and challenges that we can glimpse but certainly not predict.” This is a pretty good overview of the kind of leaps we will have to make technologically, but most of all it is that same assumption that Kurzweil and the other geek rapture folks are counting on: that these technologies will converge at all.
However, apart from the overall technological trend that must occur, we also run into some problems we did in the previous paper, essentially in the human compassion and understanding department. Wallach says that “A few such ethical assistants have already appeared. Apache Medical Systems has produced a series of decision support software tools that provide real-time, risk-adjusted clinical and financial information that helps physicians and hospital administrations manage care for high-risk, high-cost patients (Knaus 2002).” This seems to me to be a case of the popular assumption that medical robots are always ethical, which in this case is totally off base. How he could call this an ethical assistant is beyond me, seeing as how even Wallach’s short description paints it as a calculating accountant, a great application for massive uncaring computation, but certainly not a moral system.
The real problem with implementing strong AI morality, or even morality simulacra in a home environment is that people barely trust other people in their home. Generally a person needs all sorts of qualifications, be it a nursing degree, a pest control company, or a close friendship to be allowed into such private space. A roomba isn’t going to rob your house or abuse your trust. It simply doesn’t have the capability, but if something has the capacity to truly act upon you in your home, I doubt anybody would put any real trust in a machine morality, no matter how computationally powerful it is.
2) b) Learning has the same problems as the hard-coded morality issue. A machine by virtue of being a machine will not store information like a human, and if it does, you are talking about simulating an entire human, which is again a mightily lofty position to take. Learning makes more intuitive sense because we trust things that have learned their lesson and can be reprimanded for their actions, but the amount of fakery and difference between robotic systems makes for a totally different social dynamic and reward-punishment system. Imagine a family building a wall of chairs around the bathroom to stop the robot from drinking from the toilet, effectively convincing it that the room does not exist. Yes, learning might be a way to make people more comfortable with robotic systems (Baxter?) but it carries the same fundamental issue that programming morality does. It’s just a different interface.
Nico said:
ALSO THE ROBOT THAT IS MEANT TO KNOW ITSELF THROUGH MIMICRY IS CALLED NICO WOAH GUYS
Max Korein said:
Wallach1
1. Wallach argues about a future in which computers have comprehensive rationality; that is, they do not suffer from the bounded rationality characteristic true of today’s humans and robots. I would argue forcefully that this is a seriously flawed argument, and that embodied robots will have a significant bounded rationality problem. First, agree or disagree with me and explain why in concrete terms. Second, if I am in fact right that this is a flawed perspective, explain the impact of this changed assumption on the key points of this article regarding artificial moral intelligence.
This almost feels like too simple a counterargument, but even we subscribe to the continuation of Moore’s law and believe that computers will be exponentially more powerful than they currently are, and even more powerful than human brains are, within a few decades, I still have my doubts about them having the computational power to exhaustively consider and accurately predict the outcomes of all possible courses of action in any ethical dilemma. There are simply too many possibilities, and, of course, if we really want to nitpick, we have to take issues like chaotic systems and quantum physics into account. The prediction of outcomes may literally be impossible, and without exhaustive prediction, I’m not sure we can ever claim the computer’s rationality is comprehensive.
Admittedly, Wallach says this will occur within a bounded context with a set of constraints, but in that case the question arises of how we can define those contexts and constraints. In a way, this feels like just an indirect way of ending up with a different satisficing solution. Instead of using a heuristic to choose an action without exhaustively considering all possible actions, we’re using a heuristic to reduce the set of possible actions to one that can be exhaustively considered, but there’s still a heuristic being used and as a result it’s still not comprehensive.
One of the article’s key points is essentially that computers may be able to overcome their lack of moral intuition and prove morally superior to human through raw processing power, similar to how they win at chess. The problem is that this assumes the amount of raw processing power required to achieve this goal will exist at some point in the future, and I think there may be doubts about this.
Wallach 2
Consider a home care robot for an elderly person. It helps with simple chores, like cleaning the floor, but also keeps its eye on the occupant. In cases where the person is having trouble- say, spending too long in the bathroom, it has the ability and option to reach out to the occupant’s children by contacting them. It has to weigh privacy concerns with maintenance of safety and health.
1. The top-down strategy that Wallach describes on pages 466-467- in computer science and AI terms, what are the key breakthroughs we would need in order to implement such top-down moral reasoning for our home care robot? How many years away do you think this is? – explain your estimate.
I think this depends a lot on the type of rules we were going to implement in our top-down moral reasoning. It might very well be possible to come up with a set of rules that the robot could follow now. Give the robot the ability to calculate both the risk and the privacy concerns of any situation, and give it a strict set of rules about under what conditions it should contact the occupant’s children in terms of the risk and privacy values of the situation. Calculating risk and privacy values might be difficult, depending on what sensory abilities it has, and I’m not sure we have anywhere near the technology right now for the robot to do them particularly accurately, but given the right set up of sensors we might be able to create a set of heuristics that would allow the robot to make not-too-terrible decisions about when to contact the occupant’s children.
So if we’re allowing a fairly significant budget for the robot’s sensing capabilities and a very quantified, utilitarian-esque set of rules, I’m not sure if a huge breakthrough would be necessary to implement something like this or not. Of course, at this point, I’m not sure if we could accurately say the robot is performing moral reasoning either. It’s just plugging numbers into a formula people wrote. It’s sort of the moral equivalent to the Chinese room, where the robot just converts input into actions with no understanding of why. Of course, most of the counterarguments against the Chinese Room scenario could probably be used here as well.
So how many years away is this home care robot that can take a top-down strategy to determining when the risks of a situation outweigh the occupant’s need for privacy? It depends on how complex we believe the rules need to be for making that determination. We could devise a simple set of rules for which the primary challenges would be sensory, and I don’t know enough about the limits of current sensor technology to say how long that would be. If we were to attempt to devise a more complex set of rules – say, have the condition that they be given to the robot in the same sort of English-language description you would give to a human caretaker – then the technology could very well be decades away.
2. The bottom-up strategy that Wallach describes on page 467- restate just how is this robot becoming a moral reasoning system? Wallach suggests that this bottom-up approach has promise due to its embedded learning nature, so that morality is learned in a context of existence and action relevant to one’s own personal experience. Yet at the bottom of p.467 Wallach suggests porting the resulting, learned system from one robot to another (in effect, a cloning of moral reasoning). What technical challenges do you foresee in this bottom-up approach? How many years away do you think this may be?
I’m not sure if this is a technical challenge, necessary, but I think one of the biggest concerns about the bottom up approach is the dangers that occur while the robot is still learning its morality system.
Let’s imagine taking a “bottom-up” approach to an autonomous car learning to drive. We can start out putting it in a setting where it can’t harm anything – maybe an empty parking lot, or the middle of the desert, or even a simulation – but eventually, it’ll have to be put in a situation where it has the potential to cause harm for the first time. And unless we have the ability to perfectly simulate every scenario, at some point, it will encounter a dangerous situation that it has no preparation for, and make the wrong decision, and there will be consequences. It can learn from those consequences that it made the wrong decision, and avoid making that mistake in the future, but it’s too late to reverse what already happened.
A bottom-up morality has the same danger. A robot that’s learning morality from experience will naturally perform many immoral actions as it occurs. And that can be a serious danger.
Granted, this happens with humans too. Given that people often learn things “the hard way”, we probably need to accept that robots with bottom-up morality would too. But there are a number of factors that stop it from being as dangerous as it could be in people. First, we’re very good at extrapolating, something most current machine learning algorithms are terrible at. This is an issue that comes up in vision: a person who’s never seen an elephant can see a single picture of one, and then identify elephants with near perfect accuracy in a set of other pictures they’ve never seen before. State-of-the-art computer vision algorithms need vastly more training data than that. The same concern could come up with morality.
Another thing is that humans, when young, have very limited capabilities. The damage a five-year-old without a strong sense of morality can cause is very limited. An adult can do far more damage, but by the time someone is old enough to make decisions with severe consequences, they will (hopefully) have learned a much stronger moral grounding. It’s possible that robots that learned morality “bottom up” would need their capabilities artificially limited before their sense of morality had fully developed.
Finally, humans have both a moral intuition and guidance from their parents. We don’t learn morality purely bottom-up. Most of us have instinctive notions of certain things being right or wrong, and if we don’t then out parents will tell us.
So getting robots to overcome all of these different obstacles and actually be able to learn morality bottom-up safely seems like a considerable challenge. I would say it’s far enough away that it’s not in the foreseeable future.
Jessica said:
I agree with the author on the following point: in the case of a “bounded context”, where the courses of possible action are well-deliniated and programmable, a “robot” could be a “moral agent”. In fact, by that definition, we already incorporate “moral agency” into our software: MS Windows implements DRM, Google’s crawler respects robots.txt, and smartphone apps are designed so they are up-front about how they interact with the user’s data.
However, I think that using the word “morality” here is extremely misleading, and in fact is totally different from the human-like (or better-than-human) “unbounded morality” mentioned elsewhere in the article. I agree with you that this “unbounded morality” is impossible for either human or robot.
What is “unbounded morality”, according to the author? It seems to be a stoic act-consequentialism on a massive scale, a Deep Blue-like program that explores possible future paths as the consequence of a given action far into the future. Yet it is impossible for a computational system, regardless of the size, to completely describle all possible consquences of a course of action–this universe is infinite. So one has to draw the line somewhere, which means that at some point the robot’s software must say, “good enough” and then act. Humans have this same constraint, and it is why we, who codify rule/policy-consequentialism in our legal system, also have built in both a) ways of interpreting the existing rules on a case-by-case basis and b) ways of changing existing rules. The “moral” robot described in the article is more of a sociopathic police officer than a human-like moral agent.
Without the possiblity of a future “unbounded morality” program, this article is worse than useless–it is dangerous and misleading. Researchers who believe in it will be led on a wild goose-chase, and the layman who believes it will put too much faith in their robotic counterparts.
This author makes an argument that I have seem many times. It goes something like the following: we need to develop artificial morality so that service robots, which interact with humans in an intimate way, can behave appropriately. As described above, an “unbounded”, “better-than-human” moral system is impossible. If you create a “good-as-a-human” moral system then you’ve created a human (which we already know how to do by the way) and you must ascribe it the agency and rights of a human. Is that a worthwhile goal? I don’t think so. I think the answer to creating better service robots is not “morality”, but predictability. Unlike robots, humans are extremely adaptive, and will learn how to function in the context of any robot that behaves predictably. Service robots should be predictable and accountable, because at the end of the day their agency derives from their human creators and owners.
————————-
1.
The problem of determining the best course of action within in a specific framework is easy — robots already do that all the time using existing search and optimization algorithms. The difficult part is creating the framework — how does the robot take unstructured data from the real-world, pick out the relevant information and describe that within the framework? For example, in the case of the home care robot: how does it know that the human is in the bathroom? how does it distinguish between “taking a long shower” and “leaving the water running unattended”?
I think that, for very simple and structured cases, we’re not very far off from implementing this approach. I’m thinking of something like an expanded Life Alert-type device (“Help! I’ve fallen and I can’t get up!”) that monitors for a wider range of scenarios and can act accordingly on its own. Once a device like that is already established in people’s homes, its feature set would naturally grow, as directed by the users, in a roomba-like way. (We’re talking about a consumer product here, and the features of these products are driven by what the end-user wants.)
2.
In this case, the robot becomes a moral reasoning system as described by its moral education software and its training environment. That is, if the training environment praises the robot for a certain action, then that action is morally good.
Problems with porting: As with machine learning software today, the model fits the training data. So if the world of training scenarios is too small, then the system overfits and would not be robust to novel environments. So, while this porting might make sense if you’re simply replacing a robot in the same environment, it would be more difficult to bring that robot’s software into a different environment than where it has been training.
I think this sort of approach is much further off. The machine learning field is still fairly new, and researchers are still working on relatively focused and structured tasks like image recognition. More importantly, the codification of a moral system is difficult–as the author mentions, over 3000 years we haven’t quite figured it out. So, unlike with the structured and pre-existing world of image processing, researchers/philosophers would have to a) narrow down an ethical system to the barest components needed for the robot’s software to function, b) determine how the robot could “sense” relevant data within this system, and c) develop scenarios for training within this system that are broad enough to be useful.
Talha Rehmani said:
I would say that future computers will definitely suffer from the bounded rationality and the methods that Wallach suggested has a lot of flaws in it. First, I disagree with the notion that emotions can be incur in a robot if we can somehow couple their vast information gathering capabilities with “biochemical and physical activity” and nervous system. Really? A human rationality characteristic is more complex than this and even if we think that this is correct, we know that it is almost impossible to make a bio-chemical or nervous system as good as human beings. We are talking about billions and billions of neurons and other human systems with their amazing processing power. I also think that human beings are more than a machine which gathers information, and in any given time our thoughts are closely couple with thousand and millions of other thoughts which may be not in active state but they are there in our conscious or subconscious and they help us in making a decision or do other chores of the life. It is more about the way we deal with information and I think it is impossible for the robot to reach to this state.
It is an interesting problem but there is probably no good solution to this. The method that I am suggesting is where you can compromise the privacy of the user when needed. It means that in the governing rules of the system, user safety has the highest weightage. So in a situation where a user is taking too much time inside the restroom, The robot maybe try to knock on the rest room door or try to communicate with the user after some pre-specified time. It should repeat this action a couple of times then if it does not hear back from the user it turn on the camera in the restroom which can help the robot understand the situation of the user and if user needs help the robot can call 911 or relatives. I understand that having a camera in the restroom is disturbing, but if the human safety is the core concern then probably it is okay to have it. In fact, if the user knows that the robot would turn on the camera, if it does not respond to its knocking or calling, then he will make sure to reply to those calls if he is a sound state. If we build something like this which compromises human privacy then maybe we will have such robots in the next decade. But if we are trying to have AMA’s then I think it’s almost impossible or maybe we need 50 years or so to even get closer to the idea of having a human kind of rational robot.
I liked the bottom-up strategy of training the robot by praising or rewarding them for their actions. It truly seems better than the top-down strategy as every user would have a robot which works according to their ethical framework. It is like growing up a baby in your household. But I have some technical and ethical concerns. Like, can we really have a robot with unlimited training capabilities? Would it work in any unstructured environment? What if the person itself is an unethical person and he trains the robot? What if somebody hacks this robot and makes a copy its “learned system” and use it against me? I think we have to consider these questions when we build systems like this which are rational and have to work very closely with the human beings. For the time period I would say, we are looking at 60 or 70 years.