*2.1. Human-in-the-Loop Techniques*

Human-in-the-Loop ML has the goal of increasing the accuracy of a ML model, reaching the target performance faster, combining human and machine intelligence to maximize

**Citation:** Bobes-Bascarán, J.; Mosqueira-Rey, E.; Alonso-Ríos, D. Improving Medical Data Annotation Including Humans in the Machine Learning Loop. *Eng. Proc.* **2021**, *7*, 39. https://doi.org/10.3390/engproc 2021007039


Academic Editors: Joaquim de Moura, Marco A. González, Javier Pereira and Manuel G. Penedo

Published: 19 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

accuracy, and assisting human tasks with machine learning to increase efficiency [1]. The most relevant tasks mentioned are:


Depending on who is in control of the learning process, we do identify different approaches: Active Learning, Interactive Machine Learning, and Machine Teaching.

#### *2.2. Active Learning*

One of the first techniques is Active Learning (AL) [4], where the system remains in control of the learning process and treats humans as oracles to label relevant unlabeled data. It is particularly useful when the labeling example process is expensive or timeconsuming, and it also applies to the scenario of scarcity of examples (e.g., cancer). AL uses an interactive/iterative process for obtaining training data, unlike passive or classical learning, where the data is provided in advance. The learner requests information from the oracle, that it selects based on different query strategies.

#### *2.3. Interactive Machine Learning*

Another approach is Interactive Machine Learning (IML), in which there is a closer interaction between users and learning systems, with people iteratively supplying information in a more focused, frequent and incremental way compared to traditional machine learning [5,6]. In this technique the learning process control is shared between the system and the users, working closely to benefit from each other.

#### *2.4. Machine Teaching*

Finally, Machine Teaching (MT) [7,8] where the idea is to focus on the teacher role a human can play to create useful information from the data available. With the aim of facilitating the construction of new models that nowadays require practitioners with deep knowledge of machine learning, this method proposes to decouple knowledge about machine learning algorithms from the process of teaching. The human would behave as a teacher guiding the learning process [9].

A particular version of MT is *Iterative* Machine Teaching (iMT) [10] whose goal is to obtain the optimal training set given a machine learning algorithm and a target model. The idea is to learn a target concept with a minimal number of iterations using the smallest dataset.

#### *2.5. Applying and Interpreting the Results*

Once the model is deployed and it is used in a production environment, we could use Explainable AI (XAI) [11] to make the results of AI systems more understandable to humans.

There are specific domains where the aforementioned methods could fulfill the targets of the expected model. As an example, ML-approaches can be of particular interest to solve issues in Health Informatics, where we are lacking big data sets, we need to deal with complex data and/or rare events, and traditional learning algorithms suffer due to insufficient training samples [2].

#### **3. Results**

To date, we explored two of the techniques exposed: Iterative Machine Teaching (iMT) and Active Learning (AL). We have analyzed how to integrate them in the learning process using common datasets: Gaussian, MNIST and Vehicles.

Our proposal to incorporate iMT and AL into the machine learning loop is to use iMT as a technique to obtain the "Minimum Viable Data (MVD)" for training a learning model, that is, a dataset that allows us to increase speed and reduce complexity in the learning process by allowing to build early prototypes.

The results of the application of the iMT and AL on known datasets can be found at [12]. There we can see that, in the iMT experiment, the results show—both in the example problems and in the real-world problem—that the algorithms trained by any of the proposed teachers obtain better results than those trained by randomly choosing the examples. In our AL experiment, we find that the greatest advantage of this approach is in the continuous improvement of the model, which enhances resilience and prevents obsolescence.

#### **4. Discussion**

The quality of the data is a key factor that can make the model to fail in certain scenarios. If our data is better our algorithms will generalize better. This is the idea of the so-called data-centric approach which is behind some of the techniques explored (i.e., Machine Teaching).

The methods described in this paper are not mutually exclusive, so they can be combined with the aim of obtaining better results. Some of the techniques apply at different stages of the ML pipeline. Furthermore they can be incrementally implemented enhancing the model at every step.

The outcomes of the experiments conducted were obtained using common datasets as inputs. Even if they are promising, we plan to apply these techniques to relevant medical databases as The Cancer Genome Atlas Program (TCGA).

As for future work, we would be interested in applying these techniques considering multi-class problems and utilize the TCGA datasets.

#### **5. Conclusions**

The techniques exposed (combined or individually) can be applied to a specific domain (Cancer diagnosis and prognosis) making Machine Learning (ML) methods accessible to subject-matter experts and improving the performance of both the system and the human (i.e., HITL-ML), obtaining semantic and interpretable ML models (i.e., Explainable AI).

**Funding:** This work has been supported by the State Research Agency of the Spanish Government,112grant (PID2019-107194GB-I00/AEI/10.13039/501100011033) and by the Xunta de Galicia, grant113(ED431C 2018/34) with the European Union ERDF funds. We wish to acknowledge the support114received from the Centro de Investigacin de Galicia "CITIC", funded by Xunta de Galicia and the115European Union (European Regional Development Fund- Galicia 2014-2020 Program), by grant116ED431G 2019/01.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.


**Iker González-Santamaría 1,2,\*, Minia Manteiga 2,3 and Carlos Dafonte 1,2**


**Abstract:** The aim of this work is to search for evidence of close binary stars associated with planetary nebulae (ionized stellar envelopes in expansion) by mining the astronomical archive of Gaia EDR3. For this task, using big data techniques, we selected a sample of central stars of planetary nebulae from almost 2000 million sources in an EDR3 database. Then, we analysed some of their parameters, which could provide clues about the presence of close binary systems, and we ran a statistical test to verify the results. Using this method, we concluded that red stars tend to show more affinity with close binarity than blue ones.

**Keywords:** astrometry; binary stars; Gaia EDR3; planetary nebulae

#### **1. Introduction**

Planetary Nebulae (PNe) are the stellar objects that are generated when low- and intermediate-mass stars eject and ionize the envelope that surrounds them, reaching their final phase of evolution. In some cases, they come from a binary star system, instead of being generated by a single star. These cases are of special interest, as they can provide information about the morphology, formation and evolution of the PNe [1].

Therefore, the aim of this work is to search for binary stars in PNe, concretely close binary systems, which should have more influence in the PNe than the wide binaries. In these cases, the closeness between both stars allows for mass transfer between them, and this effect could generate a stellar structure made of gas known as a common envelope. This would be the origin of some peculiar PN morphologies, such as bipolar ones (see Figure 1).

**Figure 1.** Common envelope between both stars in a close binary system.

To carry out this research, we relied on data provided by ESA's Gaia satellite, which was launched at the end of 2013 with the aim of making a star map of the Milky Way.

**Citation:** González-Santamaría, I.; Manteiga, M.; Dafonte, C. Close Binary Stars in Planetary Nebulae through Gaia EDR3. *Eng. Proc.* **2021**, *7*, 40. https://doi.org/10.3390/ engproc2021007040


Academic Editors: Joaquim de Moura, Marco A. González, Javier Pereira and Manuel G. Penedo

Published: 19 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

We made use of the recently published (December 2020) data archive from this mission: Gaia EDR3 (Early Data Release 3). This database contains astrometric and photometric parameters of almost 2000 million stellar objects, so its exploitation requires the use of big data techniques.

#### **2. Materials and Methods**

The first step was to select a galactic PNe sample on which to analyse the binarity of their Central Stars (CSs). To carry out this data-mining process, we created a crossmatch between the PNe coordinates from the literature [2] and sources from the Gaia EDR3 archive. Through this procedure, we obtained a 2035 PNe sample with reliable CS identifications.

The detection of close binary stars is not an easy task, because the proximity between both stellar components means that they cannot be visually resolved as two separate sources. However, it is known that the presence of a companion star can influence the decrease in the astrometric data quality that Gaia measures for the CS. Consequently, to find some evidence of binarity, we analysed certain parameters of the EDR3 database that were related to this detection quality. One of these parameters was astrometric excess noise, which measures the disagreement between the observations of a source and the best-fitting standard astrometric model. Another significant parameter was the Image Parameter Determination (IPD) harmonic amplitude; this measures the deviation in the image centroid fitting. Other parameters that could be considered are uncertainties in the source coordinates, in Right Ascension (RA) and in Declination (Dec) units. We also included the Renormalised Unit Weight Error (RUWE) parameter, which is related to the goodness of fit to the models when a source is detected. As the Gaia satellite is scanning the sky constantly, it takes measurements throughout multiple epochs for each source during its mission. Therefore, the mentioned parameters were calculated by data-mining techniques, considering and processing all the data collected from different epochs.

In addition, the colour of the star could also shed light on the presence of a possible binary system. The CSPNe usually have blue colours (indicators of a high temperature, capable of ionizing the PN). Therefore, stars detected as having reddish colours could be related to the presence of a companion star that is overshadowing, and consequently reddening, the central star. Therefore, with the aim of analysing this possible effect, we decided to separate our PNe sample into two subsets: blue and red stars. Then, in order to independently analyse their relationship with binarity, we calculated the mean values of each Gaia EDR3 parameter given for each subset.

#### **3. Results**

As a result, we found that the subset of red stars tends to have higher mean values in these quality parameters than the blue stars subset. This means that the quality of measurements is worse in the case of red stars.

We obtained much more astrometric excess noise for red stars than for blue ones, while the IPD harmonic amplitude showed similar values for both subsets, interms of mean. Regarding the coordinate uncertainties and RUWE parameter, we also obtained slightly higher values for red stars than for blue ones. Therefore, red CSPNe would have more probability of belonging to a binary system. In Table 1, the mean values for each parameter in each subset can be observed.

Furthermore, with the aim of corroborating this hypothesis, we performed a Kolmogorov– Smirnov statistical test over these parameters and between both subsets, to analyse the similarity between both samples (red and blue). The *p*-values and D-values obtained from this analysis are shown in Table 1. For *p*-values below 0.1, the null hypothesis that both samples are similar can be rejected, with a significance of 99%. If the corresponding D-value is greater than 0.153, the null hypothesis can be also rejected.

Therefore, all values (except RUWE) indicate a non-similarity between both subsets. This confirms that the subset of red stars tends to have more affinity with binarity than the subset of blue stars.

**Table 1.** Mean values (with uncertainties) of different detection quality parameters for both samples (blue and red stars). In addition, the obtained *p*-values and D-values from a Kolmogorov–Smirnov statistical test between both samples and over those parameters are provided.


#### **4. Discussion**

Using Gaia accurate astrometry and data-mining methods, we were able to collect a PNe sample with reliable CS identifications, from almost 2000 million sources in Gaia EDR3 database.

Then, using this sample, we carried out a statistical analysis of several quality parameters, which enabled us to clarify which type of star has a higher possibility of forming a close binary system. In this type of system, both components cannot be visualised as separate sources, and it may be necessary to apply a statistical method to draw any conclusions.

Next year, with the launch of Gaia DR3, a greater quantity of astrometric and photometric data are expected, with increased accuracy. This will allow us to shed more light on the close binarity in PNe.

**Author Contributions:** Conceptualization, M.M.; methodology, I.G.-S. and M.M.; software, I.G.-S.; investigation, I.G.-S., M.M. and C.D.; writing—original draft preparation, I.G.-S.; writing—review and editing, M.M. and C.D.; supervision, M.M. and C.D. All authors have read and agreed to the published version of the manuscript.

**Funding:** Funding from Spanish Ministry project RTI2018-095076-B-C22, Xunta de Galicia ED431B 2021/36, and AYA-2017-88254-P is acknowledged by the authors. IGS acknowledges financial support from the Spanish National Programme for the Promotion of Talent and its Employability grant BES-2017-083126 cofunded by the European Social Fund.

**Data Availability Statement:** This research has made use of data from the European Space Agency (ESA) Gaia mission, processed by the Gaia Data Processing and Analysis Consortium (DPAC): [https://gea.esac.esa.int/archive/] .

**Conflicts of Interest:** The authors declare no conflict of interest.

