Ethical Governance of AI: An Integrated Approach via Human-in-the-Loop Machine Learning ^†

Abstract

AI technology is empowering human society. As for the ethical issues arising from technology, there have been many concerns and discussions, and the corresponding governance approaches have been put forward. However, there is not yet an integrated approach. The known governance approaches have a variety of flaws, leading to theoretical problems such as existence, interpretation, and value, and practical problems such as justice, security, and responsibilities. To build an integrated AI governance approach considering multi-stakeholder, an ethical governance approach of AI named "human-in-the-loop" is proposed in this paper. It analyzes the advantages of this method and points out future problems.

1. Introduction

AI science and technology enable human beings to manipulate a great many aspects of life effectively [1,2,3,4,5,6]. With the generation of ChatGPT and other generative AI [7], big data-driven knowledge learning [8], cross-media collaborative processing [9], augmented intelligence [10], collective intelligence, and autonomous system [11] have attracted so much attention. The new generation of AI plays multiple critical roles in our daily life [1,2,3,4,5,6,7,8,9,10,11]. Correspondingly, the diversity and uncertainty of future AI development will bring new challenges to its ethical governance as well, and it is urgent to ensure the credible, controllable, and sustainable development of AI [12,13,14,15,16,17,18,19].

To create an AI that is more responsible than current, achieving effective ethical embedding should be the primary mission [20]. In the technical process, the key step of ethical embedding is data annotation. There are two approaches, one is the top-down human annotation (expert system of symbolist approaches, such as Bacon, MedEthEx, etc.), and the other is the bottom-up machine annotation (such as Driverless vehicles, Ask Delphi, etc.) [14], both of them have their disadvantages. The former has high requirements for the annotation, but the annotation process will inevitably be influenced by the personal will, moral cultivation, and values of the annotation. If the annotation is not ethical enough, then the machine designed will not be ethical. This will affect the fairness and consistency of the system. The latter is sensitive to the data learned by the algorithm, but the machine learning process is affected by multiple factors such as human behavior, semantic recognition, algorithm optimization approach, and the environment, which affects the security and controllability of AI. The inappropriate data annotation approach will lead to the lack of accuracy, robustness, and comprehensiveness of AI in ethical judgment and decision-making. This will create unethical AI products. To improve that situation, this paper put forward a human-in-the-loop AI ethics embedded method, and analyzes its characteristics and feasibility. Finally, it explains why this method can be used as a new type of AI governance approach and play an important role in the future.

2. Problems

Because AI can’t make a moral judgment, it is urgent to improve its autonomy in self-recognition and scene recognition. This means breaking through the technical bottleneck of machine learning and improving its transparency and security performance. Both human annotation and machine annotation have possibilities to embed ethics and achieve “good”, but they also have corresponding flaws, which makes ethics embedding difficult. On one hand, we need the transparency and interpretability brought by human annotation to expect AI to operate stably and continuously. On the other hand, we also need the flexibility and efficiency brought by machine annotation to adapt to the complex ethical context. In one short, we need to thoroughly combine human intelligence with machine intelligence.

At present, although AI has been proven to achieve the enhancement and expansion of human ability to a certain extent, humans know little about the various possibilities of machine intelligence. The different values and demands of different stakeholders also mean that a universal ethical framework is difficult to establish, and the development of this technology is increasingly out of control. Therefore, as active moral agents, human beings should assume the responsibility of prediction, design, and supervision for the development and management of AI technology. It is necessary to explore the appropriate AI ethical embedding method (annotation method) and governance approach, to promote the integration of human-machine value and establish a comprehensive framework for AI ethical governance (Figure 1).

3. Solution

Human-in-the-loop is a semi-supervised human-computer interaction learning method [21], which aims at achieving the accuracy of machine learning and assisting human learning. It implements the high efficiency and accuracy of AI in decision-making through scene recognition, high-quality human annotation, and active learning. A possible ethical solution can be provided by using the steps as follows (Figure 2):

• Scene recognition: For a certain scene of AI decision-making, try to design an online questionnaire, and open it to the public to collect their views;
• Marks: Mark the original data of step 1, use machine learning to mark some of the processes of semi-automation, or help to improve the efficiency. In the context of ethical decisions, the best practice is to implement the " loop" and annotate the data correctly, using the annotated data to train the model, then use the trained model to sample more data for labeling;
• Active learning: Combining the sampling strategy of diversity, uncertainty, and randomness, the results of the online questionnaire in step 1 were sampled and analyzed, and adjusted to the sampling strategy of active learning. The iterative process in step 2 is repeated until an artificial decision-making process that is most close to a real-world scenario is generated and ethically justified.