Author: Zhao Leile (Associate Professor, School of Education, Jiangnan University); Dai Ruihua (Professor, School of Education, East China Normal University, Director of the Training Center for Secondary School Principals, Ministry of Education)

Artificial intelligence is becoming a new focus of international competition and also brings new development opportunities to human society. The rapid development of technologies such as machine learning, natural language processing, affective computing, and virtual reality not only provides strong support for students to obtain higher-quality educational experiences, but also helps to further expand teachers’ space for free development. But at the same time, artificial intelligence technologies such as intelligent tutor systems and intelligent data mining face many ethical issues in educational applications, such as data leakage, public safety, and malicious competition.

Artificial intelligence (, AI for short) is a part of computer science, including the research and development of theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Educational artificial intelligence ethics is used to standardize the principles and principles of relationships and behaviors in the development and application process of educational artificial intelligence.

Although educational artificial intelligence has powerful data integration and analysis capabilities, it will also trigger a series of new ethical risks, which are mainly reflected in the following four aspects from a systematic perspective.

First, information security. While artificial intelligence assists education and teaching, it not only creates opportunities for the development of teachers and students, but also generates massive educational data such as personal identity information, behavioral trajectories, learning preferences, etc. If such data is not effectively protected or used illegally, it will cause a series of information security issues, such as malicious leakage and improper use of personal privacy information, malicious push of emails, illegal electronic monitoring, online fraud, data infringement, etc. The privacy rights of teachers and students will become very vulnerable under the lens of artificial intelligence.

Second, there is a conflict of goals. The risk of goal conflict in educational artificial intelligence is mainly caused by the inconsistency in the basic goals of artificial intelligence developers and users. For the private sector, due to the deep-rooted concept of putting commercial interests first, it may rely on artificial intelligence technology tools to excessively pursue self-interest. Moreover, many designers and developers of educational artificial intelligence lack the sense of responsibility to be committed to "educational public services" and lack the value concern for educational growth. If artificial intelligence developers and school management teams fail to reach a consensus on the educational application of artificial intelligence, it will be difficult to effectively protect the interests of teachers and students in the long term.

Third, there is a lack of institutional constraints. Currently, the institutional system for restricting the development and application of educational artificial intelligence is not yet complete, and it is difficult to effectively protect the legitimate rights and interests of educational users. Moreover, the absence, dislocation and dislocation of the professional institutional restraint system have become important factors leading to the emergence of unethical behavior in educational artificial intelligence. In the absence of effective technical standards and "mandatory" legal means to regulate it, the design, development and practical application of educational artificial intelligence may gradually deviate from educational values ​​and humanistic stances. Accountability and sanctions should be implemented for behaviors and measures that violate ethical standards for educational artificial intelligence, so as to reasonably regulate the behavior of artificial intelligence developers, educational users, and others.

Fourth, excessive resource dependence. At present, although machine learning and algorithm recommendations can achieve personalized push of teaching resources, this seemingly fair and reasonable artificial intelligence application implies unknown algorithmic discrimination and technical bias, which may lead to a high degree of homogeneity in the teaching resources obtained by students. Students gradually become dependent on resources and are unwilling to break away from rigid thinking, which in turn affects the broadening of students' horizons and innovative development. Moreover, teachers may also rely too much on teaching resources pushed by artificial intelligence in order to save their own time and work costs, and lose their own unique understanding and thinking about education and teaching.

Clarifying the path to eliminate relevant ethical risks will help enable education to make ethical use of artificial intelligence opportunities and achieve fair and ethical educational decision-making.

First, focus on cultivating the information literacy of teachers and students in the intelligent era. First of all, it is necessary to pay attention to the cultivation of teachers and students' information discrimination ability. Schools should guide teachers and students to understand and think about information resource acquisition and screening, information ethical risks and other issues, and enhance teachers and students' information discrimination abilities. Secondly, pay attention to the cultivation of information feedback ability of teachers and students. Schools should guide teachers and students to provide reasonable feedback on information security, ethics, and comfort issues of artificial intelligence, and build smooth information feedback channels to continuously improve the effectiveness of artificial intelligence in educational applications.

Second, improve the review and decision-making mechanism for educational artificial intelligence algorithms. First, attention should be paid to building a review mechanism for artificial intelligence algorithms. Bias in the operation of artificial intelligence algorithms are likely to persist or even be amplified. Basic specifications and security protocols for the training and testing of artificial intelligence algorithms should be formulated, and the premise assumptions of the artificial intelligence algorithm model should be analyzed at the essential level of education. In particular, whether the educational artificial intelligence algorithm system contains problems of algorithmic discrimination and abuse should be reviewed, so as to maximize the protection of the accuracy, privacy and security of user data and information, and ethical norms. Secondly, the decision-making mechanism for artificial intelligence education application plans needs to be clarified. Governments, schools, enterprises, etc. all need to adhere to the decision-making principle of brainstorming, encourage the presentation of different opinions and open discussions, consider some form of voting and evaluation to determine the optimal decision on the application of artificial intelligence in education, and strive to achieve the reasonable satisfaction of public interest demands in education.

Third, strengthen regulatory supervision of artificial intelligence and data. The current development of artificial intelligence in education has exposed many technical and data anomie problems. Therefore, it is necessary to improve the risk accountability mechanism for intelligent technology and educational data and achieve standardized supervision of intelligent technology and data. First, we can try to formulate and promulgate legally binding educational artificial intelligence regulations to standardize the ethical responsibilities of technology development and application. Illegal development, use and sales of artificial intelligence will face legal consequences and liability. Secondly, focus on data supervision in the application process of educational artificial intelligence and build professional data privacy supervision organizations or institutions. Privacy regulatory organizations or agencies should evaluate the data security of a certain artificial intelligence technology before promoting its application to avoid illegal sharing and use of data.

Fourth, strengthen the investigation and assessment of ethical risks in educational artificial intelligence. To achieve ethical educational artificial intelligence, we cannot rely solely on technological improvements. We should also use "systematic thinking" to examine and investigate the ethical risks of educational artificial intelligence. This is crucial to maximizing the protection of the public interest in education. The existence of ethical risks in educational artificial intelligence involves not only the development stage, but also the production and distribution stages. In addition to ensuring the safety of educational artificial intelligence design, the hazard levels of intelligent algorithm ethical risks, educational data ethical risks, etc. should also be carefully evaluated in order to timely formulate educational artificial intelligence ethical risk resolution plans.

Fifth, build a school-based education artificial intelligence ethical standard system. The school-based educational artificial intelligence ethical norm system must be highly fair and interpretable, clearly stipulate the basic requirements, review procedures and ethical norms for the application of artificial intelligence in schools, and reasonably divide and limit the responsibilities, rights and obligations of technology developers, school administrators and others in the ethical supervision of educational artificial intelligence. School administrators can set permissions for the application of artificial intelligence in school education based on the school-level education artificial intelligence ethics system. If a certain artificial intelligence technology creates significant ethical risks in its application, the application permission of artificial intelligence should be appropriately restricted based on the assessment level of ethical risks.

(This article is part of the 2020 special key project of the China Higher Education Association "Research on the Prevention and Mitigation of Information Security Risks in Colleges and Universities"

results)