Kilonova-Targeting Lightcurve Classification for Wide Field Survey Telescope

Abstract

With the enhancement of the sensitivity of gravitational wave (GW) detectors and capabilities of large survey facilities, such as the Vera Rubin Observatory Legacy Survey of Space and Time (LSST) and the 2.5 m Wide Field Survey Telescope (WFST), we now have the potential to detect an increasing number of distant kilonova (KN). However, distinguishing KN from the plethora of detected transients in ongoing and future follow-up surveys presents a significant challenge. In this study, our objective is to establish an efficient classification mechanism tailored for the follow-up survey conducted by WFST, with a specific focus on identifying KN associated with GW. We employ a novel temporal convolutional neural network architecture, trained using simulated multi-band photometry lasting for 3 days by WFST, accompanied by contextual information, i.e., luminosity distance information by GW. By comparison of the choices of contextual information, we can reach 95% precision and 94% recall for our best model. It also performs good validation of photometry data on AT2017gfo and AT2019npv. Furthermore, we investigate the ability of the model to distinguish KN in a GW follow-up survey. We conclude that there is over 80% probability that we can capture true KN in 20 selected candidates among ∼250 detected astrophysical transients that have passed the real–bogus filter and cross-matching.

1. Introduction

The merger of binary neutron stars (BNS) and neutron star-black hole (NSBH) could be the source of thermal emission extended from near-infrared to ultraviolet, which is powered by r-process-generated radioactive decay of heavy elements in the ejecta during the merger, often referred to as kilonova (KN) [1,2,3]. It is believed that kilonovas are typically fainter than supernovas and fast fading within a week  [4,5]. Often during this process, a highly relativistic jet along the polar axis could launch short $\gamma -$ray bursts (sGRB) lasting for a few seconds [6,7,8]. The interaction of the jet and interstellar medium powers the X-ray afterglow, which is spread within a relatively wide viewing angle [9,10,11,12]. The dawn of multimessenger astronomy associated with gravitational wave (GW), heralded by the detection of a compact binary merger, GW170817, and its electromagnetic (EM) counterparts AT2017gfo and GRB170817A, has ushered in a new era of multimessenger astrophysics [2,3,13,14,15,16,17,18,19,20,21,22,23,24]. With the completion of the thereafter third LIGO/Virgo observing run, O3, and a total of 90 GW candidates identified, GW astrophysics has jumped into a time-domain era [25,26]. In addition, the observation of EM signature has allowed for more accurate inclination and distance measurements of the host galaxy by model fitting and identification. In fact, EM counterparts of GWs are important sources of bright standard sirens to constrain the Hubble constant, e.g., $H_0={74}_{-8}^{+16}\mathrm{km}{\mathrm{s}}^{-1}{\mathrm{Mpc}}^{-1}$ for GW170817, which could shed light on the Hubble tension problem [27,28].

In the span of the O3 run, a total of 56 public alerts were released by LIGO/Virgo through the gamma-ray coordinate network (GCN) notices and circulars1 [25]. While extensive prompt follow-up observations were conducted following low latency public alerts, which yield hundreds to thousands of candidates, no confirmed kilonova was identified [29,30]. The reason for the undesirable outcome is controversial. The fast-fading signature of KN and very limited sky coverage induced by poor localization might be responsible [31,32]. However, poor sky coverage and selection criteria can also affect the data stream, which indicates that a true KN could be rejected by real–bogus classification, astrophysical origin selection, and even KN classification [30].

The ongoing LIGO/Virgo/KAGRA (LVK) fourth observation run, O4, will reach ∼160 Mpc for BNS merger detection, and over $\mathcal{O}\left(10\right)$ are expected to be detected by LVK [33,34]. The sensitivity of the O5 run will extend to ∼330 Mpc detection in the next decade, which implies the potential for discovering a BNS merger will increase by an order of magnitude. [35]. Many efforts are focusing on rapid deployment and optimization of optical follow-up triggered by GW and GRB public alerts [36], e.g., Zwicky Transient Facility2 (ZTF, [29,37]), Dark Energy Camera3 (DECam, [30,38]), Wide Field Survey Telescope4 (WFST,  [39,40]) and next generation Vera Rubin Observatory Legacy Survey of Space and Time5 (LSST, [41,42]).

The 2.5 m WFST, installed at Saishiteng Mountain near Lenghu on the Tibetan Plateau, China, will strongly support various science cases, including time-domain astronomy, asteroids and the solar system, the Milky Way and its satellite dwarf galaxies, galaxy formation and cosmology and so on [40]. With a field of view (FoV) of 6.55 ${\mathrm{deg}}^2$, it could cover ∼${10}^3{\mathrm{deg}}^2$ within a night with a 5$\sigma$ depth of 22.31, 23.42, 22.95, 22.43, 21.50 mag under 30 s exposure in five bands ($u,g,r,i,z$), respectively, making it one of the most powerful facilities in the northern sky for discovering GW counterparts [43]. In addition, excellent observation conditions with an average diopter of $0.75$ arcsec and 22.0 $\mathrm{mag}{\mathrm{arcsec}}^{-2}$ V-band background provide potential for high-quality data [44].

It is common to be overwhelmed by the data stream produced by the rapid and deep searching of wide-field instruments. Since it is not sufficient to identify KN solely by photometry, efficient KN classification is still of great significance for maximizing identified KN. Traditionally, the KN photometry classification is based on several criteria, e.g., decay rate, color evolution [29] or model fitting, and it is upgraded to a complete pipeline, e.g., ZTFReST [45]. Another method is employing a machine learning classifier, which was implemented during O3 and is well-designed so far [30]. Stachie et al. [46] adapted the long-term lightcurve RAPID [47] classifier of short-term KN detections. Chatterjee et al. [48] deployed a KN classifier, which uses a similar structure, including GW skymap information. Biswas et al. [49] designed a fast transient classification algorithm, aimed at KN, which is implemented as a module in Fink broker6, a data stream processor software for ZTF and LSST. Sravan et al. [50] proposed a fully machine-directed pipeline for KN discovery, including the optimization of the survey plan to maximize the chance of identifying KN.

Of great importance is the need for a rapid and efficient KN classifier for the WFST target-of-opportunity (ToO) observation. Thus, in this work, we simulate lightcurve data from KN and contaminants detected by WFST with designed capability, air conditions, and strategies. Subsequently, a modified RAPID framework is employed to train and test the performance of our simulated WFST ToO data generated using mock GW skymaps [35].

This paper is organized as follows. Section 2 describes the simulation of training data. We begin with the mock GW skymaps and survey plans that are generated automatically through the ToO pipeline. KN and contaminants are simulated using Monte Carlo simulations, which follow their space and time distribution, i.e., KN is distributed following GW, and contaminants are distributed following their volumetric rate. Then, we implement mock photometry and collect their lightcurves and associated information such as location and line-of-sight probability. The classifier framework and training process are detailed in Section 3. In Section 4, we test the performance on both the dataset and real data and explore the situation in simulated GW follow-up surveys. Finally, we present our conclusions in Section 5.

2. Transients Simulation

Our simulation has three steps: firstly, we randomly choose mock GW events detected by LVK with O4 sensitivity. For each event, many KN and contaminant objects are placed across the sky. Then, we trigger our KN ToO pipeline to deploy mock surveys. Finally, we collect lightcurves given certain survey plans and contextual information for each object. The full dataset is obtained by iterating all select events.

Since the aim of this work is the KN classifier for WFST during O4, the amount of BNS and NSBH events detected so far is not adequate to cover the diversity of skymaps and survey cadences. Petrov et al. [35] simulated GW signals from BBH, BNS, and NSBH mergers and employed a more realistic threshold under O3, O4, and O5 sensitivity. They gave comparable sky localization to O3 and concluded that the sky localization might be even worse during O4, which coincides with recent observations. Therefore, we randomly chose 250 GW events7 from the BNS merger as our mock events that will happen at random in 2024, in which the WFST will be online.

2.1. Survey Simulation

In practical GW observations, an accurate localization of the event plays a pivotal role in triggering ToO observations. To initiate it effectively, it is essential that the observable area for the event aligns with the survey capability of the telescope, ensuring a sufficiently high probability of observing the specific sky region. In our study, we carefully excluded sections of the localization area falling within $\pm {15}^{\mathrm{o}}$ of the galactic plane and regions with declination $\delta <-{30}^{\mathrm{o}}$ for the mock GW skymaps. Additionally, we incorporated the restriction that the airmass should not exceed two to define the observable sky area for each event. Based on the observable area and the probability of observing the sky region, we filtered 250 events, selecting those that meet the following criteria: an observable area is less than 3000 ${\mathrm{deg}}^2$ and the probability of observing the 2D sky region is not less than 0.2. Among them, 68 events finally triggered the ToO survey.

Then, we generated survey plans by implementing gwemopt8, which was originally developed by Coughlin et al. [42], serving the purpose of optimizing the EM follow-up search for GW events. During O3, several post-event observation plans were formulated using gwemopt for both individual and joint observations by multiple telescopes (e.g., [29,51,52,53,54]), yielding good performance.

In the process of generating survey plans, gwemopt includes several algorithms within each step: (1) skymap tiling, (2) time allocations, (3) scheduling. Coughlin et al. [42] extensively discussed the efficiency of various combinations of these algorithms, determining that the combination of the (1) MOC algorithm, (2) power-law algorithm, and (3) greedy algorithm yielded the most efficient results. Therefore, we employed this combination of algorithms for our simulations. Specific inputs need to be prepared before running the code, namely the merger event, exposure time, bands, and observation windows. Initially, the merger times for the GW events were randomly distributed from 1 January to 31 December 2024 to match the future operational timeline of O4. The selection of nightly coverage frequencies and bands was based on the observable area of the events and the lunar phase conditions. $g,i-$ and $r,i-$bands were prioritized around the new moon and full moon, respectively. We also adjusted our exposure time according to the luminosity distance of the event and the estimated time spent in observation. Currently, we assume an observation window for three days post-merger, during which WFST would repeatedly cover the target area. Upon the completion of the code, it will generate a list containing the corresponding pointings for each exposure, the observation time, and the cumulative probability within the coverage area as output. We show two examples of the GW skymap, MS_239 and MS_332, and the corresponding triggered WFST ToO survey in Figure 1. The tiles in the map are footprints of a one-night survey.

In Figure 2, we show the cumulative density of elapsed time between the follow-up trigger and GW trigger. The first observation time and overall observations are almost uniform within 0.8 days and 2 days, respectively. The sky coverage is also widely distributed between 0.2 and 1, indicating that we have covered as many scenarios as we can.

2.2. KN Simulation

For the KN simulation, we used two models to generate spectral energy distribution (SED). The first one is a two-component model first presented by Bulla [28,55], in which the spectra of KN were calculated using Monte Carlo radioactive transfer code POSSIS9. The first component, dynamical ejecta, is characterized by mass $M_{\mathrm{dyn}}$, which has velocities ranging from 0.08c to 0.3c. There are two compositions that are formed via different channels, in which lanthanide-rich composition is distributed within angle $\pm \Phi$ around the equatorial plane, which is formed due to tidal processes and lanthanide-free composition that is otherwise formed by a hydrodynamic interaction. The second component is post-merger wind ejecta, $M_{\mathrm{pm}}$, which is distributed spherically and has relatively lower velocities ranging from 0.025c to 0.08c representing outflow from the accretion disk. To generate SEDs of arbitrary parameters, we construct a surrogate model using a neural network following the method in Ref. [56,57,58]. The ejecta mass and velocity are related to the binary property involving mergers, such as chirp mass, mass ratio, and equation of state (EoS). Many numerical relativity simulations have analyzed the process of merger and KN explosions  [59,60,61,62,63,64]. Therefore, it allows us to bridge a connection between the BNS sample and a KN lightcurve sample.

For dynamical ejecta $M_{\mathrm{dyn}}$, we used the fitting formula from Coughlin et al. [63], which was extended from Ref. [61,65],

$${log}_{10}{M}_{\mathrm{dyn}}^{\mathrm{fit}}=\left[a\frac{\left(1-2C_1\right)M_1}{C_1}+b{M}_2{\left(\frac{M_1}{M_2}\right)}^n+\frac{d}{2}\right]+(1\leftrightarrow 2),$$

(1)

where $M_{1,2}$ and $C_{1,2}$ represent the mass and compactness of two compact objects, respectively, and fitting constants $a=-0.0719,b=0.02116,c=-2.42,\mathrm{and}d=-2.905$. $(1\leftrightarrow 2)$ represents the exchanging of subscripts. The mass of dynamical ejecta is sensitive to the mass ratio $q=M_1/M_2$ and compactness of neutron stars. We also characterize the fraction of the red component of ejecta presented in Ref. [59,66] that separate lanthanide-rich and -poor components by a threshold $Y_e$∼0.25 when $q>0.8$, whereas for $q<0.8,f_{\mathrm{red}}$∼1 because the less massive NS is disrupted by tidal forces that suppress the shock [60,67,68,69]. The fraction of the red component can be written as

$$f_{\mathrm{red}}=\min \left(1,a{q}^2+bq+c\right),$$

(2)

with $a=14.8609,b=-28.6148,c=13.9597$. Using the spherical ejecta density profile assumption in the Bulla model, it is easy to obtain the half-opening angle for lanthanide-rich components.

For post-merger ejecta, we used the following expression to evaluate disk mass [63],

$${log}_{10}\left(M_{\mathrm{disk}}\right)=max\left(-3,a\left(1+btanh\left(\frac{c-M_{\mathrm{tot}}/M_{\mathrm{thr}}}{d}\right)\right)\right),$$

(3)

with fitting parameters that include mass ratio dependence $a=a_0+\delta a·\xi ,b=b_0+\delta b·\xi$ and the free parameter $\xi$ given by

$$\xi =\frac{1}{2}tanh\left(\beta \left(q-q_{\mathrm{trans}}\right)\right),$$

(4)

where $q=M_1/M_2\le 1$. The best-fit values of free parameters are $a_0=-1.581,\delta a=-2.439,b_0=-0.538,\delta b=-0.406,c-0.953,d=0.0417,\beta =3.910,{\widehat{q}}_{\mathrm{trans}}=0.900$. $M_{\mathrm{tot}}$ is the total mass of BNS, and $M_{\mathrm{thr}}$ represents the threshold mass for prompt massive neutron star collapse to a black hole with the expression [28]

$$M_{\mathrm{thr}}=\left(2.38-3.606\frac{M_{\mathrm{TOV}}}{R_{1.6}}\right)M_{\mathrm{TOV}},$$

(5)

where $M_{\mathrm{TOV}}$ is the maximum stable mass for non-rotating NS, and $R_{1.6}$ represents the radii of $1.6M_{\odot }$ NS. The disk wind ejecta mass is assumed to be proportional to disk mass $M_{\mathrm{pm}}=f·M_{\mathrm{disk}}$ with f ranging from 0.1 to 0.5 [70,71,72,73]. In this work, we fixed $f=0.2$ as the default set. It is noted that explicit density, heating rate and opacity profile, and structured jet are considered in the new version of the POSSIS 2.0 model [74,75].

We also employed another semi-analytical kilonova model presented in Nicholl et al. [66], thereafter MOSFiT KN, which is implemented by the open-source software MOSFiT10. The model is based on the binary property forwarded to SED. They employed a numerical relativity fitting formula to convert binary parameters to ejecta parameters. The ejecta contains three components with different grey opacity, namely blue ($\kappa =0.5{\mathrm{cm}}^2{\mathrm{g}}^{-1}$), red ($\kappa =10{\mathrm{cm}}^2{\mathrm{g}}^{-1}$), and purple ($\kappa =0.5{\mathrm{cm}}^2{\mathrm{g}}^{-1}$) components. The red and blue components represent dynamical ejecta with higher velocity, and the half opening angle of red ejecta is fixed to $\Phi ={45}^{\mathrm{o}}$. This model also includes the cocoon emission and magnetic enhancement on blue ejecta mass.

Once the KN model is prepared, we can obtain the KN SED sample from a BNS sample and inject their parameters into two KN models. In the standard isolated binary formation channel, a recycled NS is born first and spins up due to the accretion or recycling process. It is accompanied by a non-recycled NS that spins down after birth. The mass of recycled NS follows a two-Gaussian distribution,

$$\pi \left(m\mid {\mu }_1,{\sigma }_1,{\mu }_2,{\sigma }_2,\alpha \right)=\frac{\alpha }{{\sigma }_1\sqrt{2\pi }}\times exp\left[-{\left(\frac{m-{\mu }_1}{\sqrt{2}{\sigma }_1}\right)}^2\right]+\frac{1-\alpha }{{\sigma }_2\sqrt{2\pi }}exp\left[-{\left(\frac{m-{\mu }_2}{\sqrt{2}{\sigma }_2}\right)}^2\right]$$

(6)

with ${\mu }_1=1.34M_{\odot },{\mu }_2=1.47M_{\odot },{\sigma }_1=0.02M_{\odot },{\mu }_2=0.05M_{\odot }\mathrm{and}\alpha =0.68{\mathrm{M}}_{\odot }$. For non-recycled NS, they are found to follow a uniform distribution within 1.15∼1.42$M_{\odot }$ [76]. Given the mass of the neutron star, we calculated the radius and compactness by sampling the parameterized EoS obtained by Dietrich et al. [28], where the EoS sample was calibrated with constraints of pulsars and multimessenger observation on GW170817. We generated the KN SED sample with a size of ∼${10}^4$ for each KN model.

2.3. Other Transients Simulation

To simulate contaminants, our main focus in this work is on supernovae, one of the most significant types of contaminants following catalog cross-matching. Still, we cannot ignore other important fast transients, e.g., cataclysmic variables (CVs), afterglows, and shock breakouts. They will be included in the complete pipeline with the aid of other selection procedures.

SN Ia. The Type Ia Supernova is thought to be powered by the re-ignition and thermonuclear explosion of carbon–oxygen white dwarfs once they exceed their critical mass [77,78]. It is the most common and numerous contaminant object to KN. To model SN Ia, we used the SED library presented by Hsiao et al. [79]. Apart from classic SN Ia, some subgroups in the SN Ia were identified through decays of observations. SN1991bg-like (SN Ia-91bg) stands out as one of the most important potential contaminants due to their bright, luminous, rest-frame $m_{\mathrm{B}}\gtrsim -18$ and fast evolution feature, with a lightcurve width of less than 70% average SN Ia [80,81].

SN Ibc. The stripped-envelope supernova explosion, also referred to as Type Ib and Type Ic supernova, characterizes the feature of lacking helium in spectra [82]. The lightcurves of SN Ibc are fainter, redder, and evolve slower, indicating they are sub-dominant sources of contamination. We used the SED template presented in Nugent et al. [83]11 to model SN Ibc, SN Ia, and its subgroups.

SN II. Type II Supernova are explosions of massive stars with mass $8\lesssim M\lesssim 18M_{\odot }$ [84]. They are distinguished from other types by the presence of hydrogen in their spectra. We modeled two subgroups SN IIn and SN IIP using the SED template in Nugent et al. [83].

SLSN-I. Type I Superluminous Supernova is one of the brightest optical transients with peak absolute magnitudes $\lesssim -21\mathrm{mag}$. They are widely distributed in metal-poor dwarf host galaxies, some of which are powered by magnetars with very strong magnetic fields [85]. Their spectra tend to be blue, lack hydrogen features, and brighten rapidly, thus making them the main contaminant in early KN identification [86,87]. To model SED, we employed the extended library of 960 SEDs from the MOSFiT slsn model by Kessler et al. [88].

2.4. Training Set

Given GW skymaps and corresponding survey strategies and transient models described above, we performed a survey simulation implemented by simsurvey12 [89], a software for survey simulation and lightcurve collection [29,89]. The simulated KN for each GW event was sampled randomly based on two KN models in Section 2.2. The explosion time of KN was set the same as the GW event, and its location was produced following the probability density of the GW skymap. For contaminants, we assume a uniform distribution of RA and Dec within the observed field of survey and specific redshift model listed in 

Table 1. Summary of transients in the training set.

Object Type	Number Count	Rate $\left({\mathbf{yr}}^{-1}{\mathbf{Mpc}}^{-3}\right)$
Bulla KN	19,694	-
MOSFiT KN	20,459	-
SNIa	13,553	$3\times {10}^{-5}(1+z)$
SNIbc	2988	$2.25\times {10}^{-5}(1+z)$
SNIIn	5492	$7.5\times {10}^{-6}(1+z)$
SNIIP	926	$1.2\times {10}^{-4}(1+z)$
SLSN	4712	$2\times {10}^{-8}$
SNIa-91bg	9180	$3\times {10}^{-6}{(1+z)}^{1.5}$

.

We randomly looped 60 skymaps and collected their lightcurves and object information, e.g., line-of-sight probability $P_{\mathrm{l}.\mathrm{o}.\mathrm{s}}$, distance mean and standard deviation $\{{\mu }_D,{\sigma }_D\}$ derived from GW skymap. It is noted that for supernovas, we truncate the distance with $z<1$, which is comparable to the detection depth of WFST. Finally, we obtained the data set containing 77,005 objects, the details of which are listed in Table 1. Since we aim to distinguish true KN among many contaminants, we labeled Bulla KN and MOSFiT KN as Kilonova and the rest as Other. Figure 3 shows the normalized distribution of line-of-sight probability, mean, and standard deviation of luminosity distance of transient according to the GW skymap of our dataset per class. One can see that the KN has a tighter distribution of features compared with contaminants because they always happen in the high probability area of the GW skymap. In addition to the lightcurve, these features could also assist us in classifying KN and contaminants.

3. Binary Classification

We employed temporal convolutional network (TCN) architecture [90], which is implemented in keras13 and RAPID14 for long-term lightcurve classification, revealing good performance. We used a similar architecture to  Chatterjee et al. [48], who modified RAPID for binary classification to filter KN.

3.1. TCN Framework

The strength and utility of the TCN architecture in the classification of a time series lie in its ability to promptly update results as new data become available. As introduced by Bai et al. [90] in 2018, the TCN architecture operates on two fundamental principles: (1) The length of the output sequence matches that of the input, and (2) the internal convolutions are causal in nature. Provided with a sequence $\{x_1,x_2,\cdots ,x_N\}$ of data, the TCN outputs a sequence $\{y_1,y_2,\cdots ,y_N\}$, and the convolutions within the hidden layers of the architecture are designed in such a way that each output, $y_n$, solely relies on the information within the input sequence $\{x_1,x_2,\cdots ,x_n\}$, where n ranges from 1 to N. Within this architectural framework, as shown in Figure 4, one can control the extent to which long-term historical information influences the output by making judicious selections of kernel sizes or incorporating dilated layers. Beyond that, one can inject the contextual information as input, which stays unchanged with time. In this work, we consider line-of-sight probability, mean and standard deviation of luminosity distance of transient according to GW skymaps and their combinations. For simulated WFST surveys with the time interval between two photometries ∼1 day, in order to reveal a more precise shape of the lightcurve, we adopt linear interpolation with a time interval $\mathrm{of}\sim$0.5 day within 5 days. Therefore, the dimensions of the input data matrix are $N\times (p+n)$, where $N,p,n$ represents the length of the interpolated time series, the number of passbands, and the amount of contextual information.

3.2. Training

For our purpose of classification for KN in ToO data, which lasts 3 days post-merger, as shown in Figure 2, we used a network of filter length $k=2$ and dilation layers $d=(1,2)$ and 2 stacks to deepen the network. We employed categorical cross-entropy as the loss function in conjunction with the Adam optimizer. For the dataset, we partitioned it into a training set comprising 70% of the data and a test set containing the remaining 30%. By testing, we found that loss tends to converge after 50 epochs of training, so we trained models for 50 epochs. Approximately 2–3 h were allocated for training and testing models. Notably, we found that the computational cost for training is comparatively modest, negating the necessity for hardware optimizations, such as GPU acceleration. Figure 5 shows the accuracy per class with time in which at least 3 days of observation yields good accuracy. The classifier tends to predict a high KN score inaccurately in instances where photometry data are lacking, as evidenced by a noticeable decline around day one. However, as more photometry data are processed, the predictions converge more closely toward the actual label.

As described above, we have three contextual datasets at our disposal to assist in classification: line-of-sight probability (hereafter P), mean, and standard deviation of luminosity distance along the line of sight of transients according to GW (hereafter M and D). To see which data affect the result most and to decide upon the best combination of information used, we consider three combinations of them: MD, PM, and PMD. Our fundamental principle relies on the mean luminosity distance defined by the GW skymap, serving as crucial guidance for the AB magnitude of transients. Therefore, the MD model and PM model lack line-of-sight probability and measurement error of luminosity distance, whereas PMD has full contextual information.

4. Results

4.1. Performance on Dataset

Using the dataset simulated in Section 2 and the TCN framework described in Section 3, we obtain MD, PM, and PMD classification models for WFST targeting KN. Among them, the PMD model reaches an overall accuracy of 98.41%. To test the performance of a classifier, it is natural to employ a confusion matrix to show their capability. In our binary classification issue, the dimensions of the confusion matrix are $2\times 2$, with true positives and true negatives residing in diagonal elements, while false positives and false negatives lie in the upper right and lower left elements, respectively. Figure 6 illustrates the confusion matrices for the MD, PM, and PMD models. The matrix values are normalized by the number of true labels, and the KN score is the last KN probability in the sequence of prediction. In the context of prioritizing the detection of KN, a threshold of 0.45 is selected to minimize the potential for false positives. Specifically, we classify a candidate as a KN when the predicted KN probability in the final epoch surpasses this specified threshold. Upon comparing these models, it becomes evident that the MD model exhibits lower efficiency when contrasted with the other two alternatives.

However, it is obviously unreasonable to use the same threshold for three different models. We estimated the performance of models under various thresholds. In the lower panel of Figure 6, we show the receiver operating characteristic curve (ROC), which illustrates the diagnostic ability of a binary classifier system. The inverted triangle curve and squared curve represent the true positive rate and false positive rate, with threshold ranging from ${10}^{-3}$ to $1-{10}^{-3}$. The threshold of 0.45 is shown as green dots in each curve. The more powerful the model is, the closer the curve is to the upper left of the coordinate. We summarize their precision and recall with thresholds corresponding to the cross of two curves, yielding 94.151% precision, 94.028% recall for the PM model and 94.830% precision, 94.125% recall for the PMD model. We notice that the PM model and PMD model have comparable capabilities while lacking line-of-sight probability is the most intolerable case.

4.2. Performance on Real Data

To further investigate the performance of the classifier, we employed similar procedures as in Chatterjee et al. [48] to test in unseen data. We considered Swope observations on AT2017gfo [91], originally identified as ‘SSS17a’ by the Swope team at the time of its discovery. It was the only identified KN accompanied by GW so far, with ejecta mass $\sim 0.05M_{\odot }$, via the fitting of various KN models [18,24,28]. We used $g-,r-,i-$band data to be compatible with our trained model input. Although the optical transmission curves of Swope are different from WFST, we maintain the belief that these disparities in the KN score’s uncertainty will not lead to misleading classification results. Additionally, we also consider a counterpart candidate of GW190814 [92], AT2019npv, which was an SN Ibc but was identified as KN in the early phase by efforts of several teams because it was located in the GW skymap, and the redshift was consistent with the distance information of GW [30,93,94]. We included the first week $i-$band observations from DECam as the WFST survey would not exceed 5 days.

In Figure 7, we implement our PMD model to predict the KN score of AT2017gfo and AT2019npv. We acquired comparable results to Chatterjee et al. [48], in which the confidence KN score is obtained from AT2017gfo and the persuasive primary KN score from AT2019npv at the early phase rather than the definitive contaminants that result from increasing observations.

4.3. Performance on Mock Survey

We have conducted thorough validation and performance tests of our model as described above. Nevertheless, it remains imperative to assess the model’s efficiency in predicting future survey data, ensuring its practical applicability. To achieve that, we applied simulations with mock surveys, as described in Section 2. In a night survey, the WFST would detect $\mathcal{O}\left(100\right)$–$\mathcal{O}\left(1000\right)$ transients ignoring history activity [40,95]. We assume that approximately a fraction of them can be filtered out by cross-matching or angular offset to their host galaxy. Therefore, we only considered cases of processing less than 300 candidates by classifier. As a comparison, an average of 170 EM candidates were filtered after multi-step machine learning, which includes the real–bogus test, cross-matching, and history exclusion [29]. To simulate candidates processed by classifiers, which have been filtered by the real–bogus test, cross-matching, and history exclusion, we simulated one KN according to the GW skymap and n contaminants with uniform distribution across the sky for each mock survey. Then, we compiled them into a data package labeled as detected candidates. The size of detected candidates is denoted as $n_{\mathrm{DC}}$. We generated five data packages for each skymap for data augmentation, considering the variety of KN models and locations. Overall, we generated 300 data packages, and we ranked KN scores in descending order in each data package, i.e., $\mathrm{rank}=1$ means the true KN has the highest KN score in detected candidates. According to our pipeline, we will take several selected candidates, the size of them denoted as $n_{\mathrm{SC}}$, for the subsequent spectra follow-up. In our analysis, we chose $n_{\mathrm{SC}}=\{10,15,20\}$ and $n_{\mathrm{DC}}\in \left(20,300\right)$. In Figure 8, we show the distribution of ranking of KN with $n_{\mathrm{DC}}=\{20,30,40,\cdots ,290\}$. The top of grey and blue bars represent the maximum and 0.8 quantiles of KN rankings with various $n_{\mathrm{DC}}$, which means over $80\%$ of KN would rank within 20 when $n_{\mathrm{DC}}$ does not exceed 290. We also calculated the 0.6 quantile, which remains 1 with any $n_{\mathrm{DC}}$.

For better clarity, we quantified the fraction of the data package in which the true KN achieved $\mathrm{rank}\le {\mathrm{n}}_{\mathrm{SC}}$ for given $n_{\mathrm{DC}}$ as the probability of inclusion. This metric reflects the classifier’s effectiveness in incorporating the true KN among the selected candidates. As expected, the classifier’s performance exhibited a declining trend with an increasing amount of $n_{\mathrm{DC}}$. Notably, we found a good fitting by a power-law function $P(n_{\mathrm{SC}},n_{\mathrm{DC}})=a·n_{\mathrm{DC}}^{-b}$ with fitting parameters $(a,b)$. The $P(n_{\mathrm{SC}},n_{\mathrm{DC}})$ curves are depicted in Figure 9. The results for the MD, PM, and PMD models are illustrated in violet, yellow, and green, respectively, and solid, dashed, and dotted lines corresponding to $n_{\mathrm{SC}}=10,15,20$. The optimal fitted parameters are outlined in

Table 2. Best fitting parameters $(a,b)$ for various $n_{\mathrm{SC}}$ of three models.

${\mathit{n}}_{\mathbf{SC}}$	MD	PM	PMD
10	1.2629, 0.1234	1.2686, 0.1060	1.2723, 0.1060
15	1.3745, 0.1332	1.3724, 0.1149	1.3552, 0.1127
20	1.5154, 0.1445	1.440, 0.1184	1.4362, 0.1165

. Leveraging this fitted curve, we can now estimate the likelihood of capturing true KN in future ToO surveys, thus facilitating our choice of $n_{\mathrm{SC}}$. For example, given ∼250 candidates detected and 80% probability of inclusion required, we should characterize $n_{\mathrm{SC}}\gtrsim 20$.

5. Conclusions

In the era of multi-messenger astronomy, especially conducted by LVK and the future 3rd generation Cosmic Explorer (CE; Reitze et al. [96]) and the Einstein Telescope (ET; Maggiore et al. [97]), it will require strongly qualified detectors to be mutually compatible, which necessitates the coordination of numerous facilities to effectively leverage the scientific prospects offered by the upcoming dataset. In light of this, the development of machine learning techniques and integrating physically informed features becomes crucial to streamlining data and minimizing the burden of human screening. The negative outcome during the LIGO/Virgo O3 run indicates that the AT2017gfo-like KN is abnormal, to date presenting substantial difficulty in identifying KN systematically. In the ∼6 months since LIGO/Virgo O4 has been running, many collaborations triggered their ToO observations, e.g., GROWTH15 and MASTER GLOBAL Robotic Net16. To enhance the probability of identifying EM counterparts of GW detections, several brokers have been developed, i.e., KN classifier embedded in Fink broker [49], EI-CIDm, which was based on the RAPID framework [48], ZTFReST [45], AleRCE [98], Lasair [99], SCiMMA17, and a fully automated pipeline for KN discovery [50].

In this work, we have presented a KN binary classifier with a modified RAPID framework. Our approach was inspired by the enhancements detailed in Chatterjee et al. [48], where the fine-tuned neural network exhibited promising performance on simulated ZTF data. We start with the simulation of transients, where we have conducted a mock WFST survey of 60 simulated GW skymaps with O4 sensitivity. The KN is located following the probability distribution of GW skymaps, whereas contaminants are located uniformly across the sky. It is noted that Andreoni et al. [45] found cosmological afterglow to be the dominant contaminant at high galactic latitude within ∼1 yr observations. Furthermore, other contaminant classes, e.g., CVs, afterglows, and shock breakouts, pose potential challenges because only a fraction of them can be filtered by cross-matching with a catalog, implying it would occasionally be a dominant source  [49]. Considering that the classifier is designed to operate on processed data ideally filtered through a real–bogus test and the exclusion of variable stars, we have simulated the majority of supernovae, which represent the predominant contaminants following the removal of these sources. Upon these foundations, we have applied three combinations of contextual information, MD, PM, and PMD, revealing comparable performances between the PM and PMD models, while the MD model proves to yield less promising results, as shown by the confusion matrix and ROC curve in Figure 6. The PMD model shows comparable accuracy, as evident in Figure 5, in which the accuracy for Other reaches $\gtrsim 98\%$ and Kilonova reaches $\gtrsim 92\%$ after 3 days since the trigger. Beyond that, we also validated models by predicting KN scores for Swope observations of AT2017gfo and DECam observations of AT2019npv, where the lightcurves and predicted KN scores are shown in Figure 7.

Furthermore, we simulated true KN accompanied by a quantity of contaminants for each mock survey to examine the performance of ToO survey data. By sorting KN scores in descending order, the distributions of rankings of true KN are plotted in Figure 8, indicating that ∼80% probability KN rankings are $\lesssim 20$ when $n_{\mathrm{DC}}$ is less than 290. In addition, we found a robust fitting to the probability of inclusion with $n_{\mathrm{SC}},n_{\mathrm{DC}}$, as shown in Figure 9, which will instruct the choice of $n_{\mathrm{SC}}$ in the future.

Through the analysis of GW observable, we found a great discrepancy between line-of-sight probability, mean, and standard deviation of luminosity distance according to GW. We did not include offset in cross-matching and $A_{90}$ of the GW skymap as in Chatterjee et al. [48] due to the uncertainty in identifying host galaxy and numerical relativity simulations. Alternatively, we explored various KN models and neutron stars, leveraging a comprehensive sampling approach across a BNS sample in conjunction with the Bulla and MOSFiT models. We anticipate retraining our models with the inclusion of data from the forthcoming WFST survey, specifically incorporating observations of real supernovae and expanding the scope of the models to accommodate phenomena such as GRB afterglows and stellar flares. Furthermore, we envision fostering collaboration among diverse observational bands, i.e., preliminary X-ray emission detectors such as Einstein Probe (EP; Yuan et al. [100]) or Chandra X-ray Observatory(CXO; Weisskopf et al. [101]), to glean additional SED features beyond the optical band.