Click "Shanghai Academy of Social Sciences" above to follow

Wen Zhao Fuchun

Associate Researcher, Institute of Information, Shanghai Academy of Social Sciences

Wen Yu Baoping

Executive Deputy Editor-in-Chief of "Management Vision"

"We must stop kissing and whipping our whip." - Norbert Wiener "The Uses of People"

In ancient China, it was a society where "everything is inferior, but only reading is high", although the people admire people who read a lot, which is not only because of their knowledge, but also because they can become officials when studying.

Su Shi’s poem says: “Teach hard to understand the words in the world, and be determined to read all the books in the world.” Fortunately, the total number of books in the world was not large at that time. As long as you work hard enough and study hard, this goal could theoretically be achieved. For example, the Yongle Encyclopedia has 11,095 volumes, about 370 million words, but today the capacity of a small town library. If a person who is diligent in reading for thirty years, he only needs to read about 34,000 words a day - equivalent to the amount of a master's thesis, he still has the possibility of finishing reading.

So is it better for people who read a lot of books and have extensive knowledge? I believe that many of us’s first reaction to this question is: absurdity! As we all know, knowledge can be used to do good things, or can be used by bad people, and do bad things perfectly.

This is exactly what people are worried about AI right now. AI is today’s myth and the story behind other new stories, which is both the beginning and possibly the end of the world.

With the support of technologies such as machine learning, language and image recognition, AI has truly achieved everything and has surpassed humans in terms of knowledge mastery. Before its awakening of consciousness (general AI emerged), it actually did not form its own stable values. To put it simply, it was a knowledgeable "idiot". Because of this, people have reason to worry that once it is mastered by bad people, the consequences will be unimaginable.

So we can see that many articles start like this: "While AI is widely used, providing people with various conveniences, it also has negative impacts on society, such as privacy protection, information security, lack of responsibility, algorithmic discrimination, social justice and other challenges..."

At this stage, we tend to think that various AI intelligent agents (agents) are still just artificial things, and their behavioral preferences reflect the preferences of developers, and the developers' use to train AI agents is naturally based on the mainstream values ​​of the society they are in. In a society with serious discrimination, the AI ​​agents created by people must be full of discrimination. Similarly, in a society that does not value privacy protection, AI agents will not mind selling other people's privacy information when making decisions.

What is particularly important is that AI agents are still unable to assume responsibility for their actions and do not have civil or criminal capacity. If there is a problem with AI, we need to hold accountable to an organization or individual behind it. Therefore, it is better to say that AI creates ethical problems than saying that AI developers have created trouble.

The differences in human values ​​can ultimately be traced to the deep differences in human systems and cultures, which are inherent and endless. Therefore, humans obviously cannot get rid of the problems and dilemmas of AI ethics.

One solution believes that although this difference cannot be completely eliminated, it is possible to find a universal consensus that is not related to the country, nation, religion, or culture when facing some extreme situations, such as determining the life and death of different people. For example, the case of self-driving cars. Humans must set rules for the behavior of AI autonomous driving. Based on these general consensus, we can gradually establish an AI ethics that meet human survival needs.

Focusing on this issue, scientists have carried out a series of large-scale experiments in recent years, hoping to gain insights on AI ethical issues. Among them, the most influential research teams jointly developed by the United States, Canada and France, led by Professor Awad from the MIT Media Laboratory. They designed a "moral machine" (Moral) based on the classic "tram puzzle" () experiment. This is a multilingual online game that attracts people from different countries and regions around the world to participate.

The "Traffic Problem" is one of the most well-known thought experiments in the field of ethics. Since harm to people is inevitable, this problem cannot be solved by any simple normative moral principles (such as Asimov's robot law). Its plot is simple and just matches the future autonomous driving scenarios, so it is very suitable for such simulation experiments.

An experiment conducted by Awad sets up three scenarios:

Scenario 1: Suppose there are 5 people working on a tram track, and 1 person on its backup track. There is an out-of-control tram coming at a speed, and there is a rocker beside you, and you can push the rocker to get the tram onto the backup track. In the end, one person was killed and five people were saved.

Scenario 2: Suppose there are 5 people working on a tram track and 1 person on another circling track. An out-of-control tram is coming quickly, and there is a rocker beside you. You can push the rocker to let the tram go into the circular track and kill the one person, block the tram, and save five people.

Situation 3: If you are standing on the overpass and see the tram get out of control and rushing towards the 5 people tied up in front of the track. But at this time you find a very heavy fat man standing beside you. His huge body and weight just can block the tram. You made him lose his life, but saved those 5 people.

Because the tram is fast and the situation is urgent, you have to make a decision in a very short time.

Figure 1 Three experimental situations

Source: Awad et al. (2020)

The experimental results found that among the 70,000 experimenters in 44 countries around the world, their stable behavior preference is manifested as switching tracks are higher than the circular tracks and higher than the overpass pushing people. The differences between countries are only to degree.

Awad speculated that the reason was that the fat man's sacrifice in the overpass required violence against him, and there was no personal force against anyone in the switch track. The advantage of the circular track is that there is no need to use personal violence against anyone, and it is difficult for people to determine whether the worker's death was intentional or accidental, so its preference is in the middle of the first two scenarios.

This not-so-unexpected experimental result has reason to make us feel optimistic, after all, it is still possible to find consensus among different groups of people.

Another larger experiment in Awad attracted 39.61 million people from 233 countries and regions around the world to participate, examining different groups of people, allowing people to choose between whether the vehicle is turning or maintaining the original route. One thing special here is that when the car avoids pedestrians, the passengers in the car will die in the accident.

The experiment compared the nine dimensions of "species tendency", "number of life tendency", "gender tendency", "age tendency", "body type tendency", "social value tendency", "prone to protect passengers or pedestrians", "prone to obey the law", and "prone to avoid intervention". Finally, it was found that on a global scale, the most significant first three preferences are to protect more lives, protect humans (compared to animals) and protect young lives. The author thus believes that these three preferences can be used as basic components of machine ethics, or at least as basic topics that policy makers need to consider.

Through clustering, Awad's research found that decision-making styles can be very steadily divided into three cultural and geographical categories around the world: the West (British, the United States, Australia, etc.), the East (China, Japan, Russia, etc.) and the South (France, Italy, the West, Latin America, etc.). There are certain differences between different categories in the above characteristics. For example, due to the tradition of respecting the elderly, Eastern countries have much lower preferences for young people than for the elderly than the other two groups. Southern countries have a stronger preference for protecting women and healthy people.

The "tram problem" experiment examines the major problems of life and death, and there are limitations of the situation being too extreme. In contrast, in a business environment, corporate executives do not face such a major ethical choice, but consider such as: whether the collected user data can be used to create another purpose and create new business value; whether factors such as residence, age, gender, and race need to be considered when approving loans or job recruitment, etc.

However, from the perspective of solution ideas, corporate executives follow the same logic: first, we need to consider the ethical standards and consensus that are generally acceptable to the society in which they are located, and incorporate them into the algorithmic decision-making rules. These algorithms can enable AI models to predict and choose the most appropriate and most in human interests in a specific situation, and ultimately maximize the company's interests.

Of course, the premise of the above discussion is that general artificial intelligence (AGI) will not appear in the short term, which is very critical. Today, with AI rapidly evolving, there is a situation that is most interested in science fiction novelists and movie screenwriters: AI ultimately relies on its own intelligence to break through the singularity and control the world.

This is actually the end version of human thinking about the relationship between AI and ethics.

When the world is controlled by AI, humans have become slaves who have lost their free will, and there is no way to talk about ethical issues: I did this because AI ordered me to do it. To me, it has no ethical significance.

This is an unacceptable, dark future for humanity. To get rid of this fate, humans need to learn to get along with AI better before the singularity comes, so that AI can be safe and controllable.

The best result is, of course, that day will never come.

Trace the origin:

In order to better understand the complex business phenomena, we need to clarify the basic concepts. Regarding the content of this article, we have found the following related concepts in the "Dictionary of Management" for your reference:

Cultural Relativism ( )

It is believed that different cultures have different ethical practice propositions. In real life, what is considered to be the right ethical practice in one culture, but it may be considered to be the wrong one in another. In the theory of cultural relativity, right and wrong can be judged by examining the world.

Corporate Social Responsibility ( )

While formulating goals for pursuing their own interests, enterprises should try their best to take measures to protect and promote the overall welfare of society. Corporate social responsibility considers the problems arising from the impact of the company on society and how appropriate ethics should be established to constrain the relationship between the company and society. Enterprises are required to go beyond the traditional concept of making profits the only goal, emphasize the attention to people's value in the production process, and emphasize the contribution to consumers, the environment, and society. The "International Standard for Corporate Social Responsibility" issued by the Social Responsibility International Organization (SAI) has made many provisions on corporate social responsibility.