A Neural-Network-Based Competition between Short-Lived Particle Candidates in the CBM Experiment at FAIR

Abstract

Fast and efficient algorithms optimized for high performance computers are crucial for the real-time analysis of data in heavy-ion physics experiments. Furthermore, the application of neural networks and other machine learning techniques has become more popular in physics experiments over the last years. For that reason, a fast neural network package called ANN4FLES is developed in C++, which will be optimized to be used on a high performance computer farm for the future Compressed Baryonic Matter (CBM) experiment at the Facility for Antiproton and Ion Research (FAIR, Darmstadt, Germany). This paper describes the first application of ANN4FLES used in the reconstruction chain of the CBM experiment to replace the existing particle competition between $K_s$-mesons and $\mathrm{\Lambda }$-hyperons in the KF Particle Finder by a neural network based approach. The raw classification performance of the neural network reaches over 98% on the testing set. Furthermore, it is shown that the background noise was reduced by the neural network-based competition and therefore improved the quality of the physics analysis.

1. Introduction

Over the last few years, the remarkable growth of machine learning and neural networks has had a profound impact across multiple scientific fields, redefining approaches to problem-solving and pushing the boundaries of scientific exploration [1]. The versatility and learning capacity of neural networks have positioned them as formidable tools in computational sciences, capable of addressing a broad spectrum of challenges across various domains. This includes, for instance, solving differential equations [2], and addressing forward and inverse problems involving nonlinear partial differential equations using physics-informed neural networks [3].

The predictive prowess of neural networks has resulted in models of outstanding accuracy. In fields like physics and computational science, these methodologies are offering new perspectives and solutions to longstanding challenges [4]. Particularly in particle physics, these methodologies permit applications at different stages of experiments, enabling entirely new approaches or complementing existing algorithms. This includes particle identification, trajectory reconstruction, event classification, and others [5,6].

This powerful interplay between computational science and particle physics will be prominently featured in the future heavy-ion experiment, Compressed Baryonic Matter (CBM), at the Facility for Antiproton and Ion Research (FAIR) [7]. This experiment will leverage advancements in neural networks for improved data analysis and interpretation. CBM is planned as a fixed-target experiment using the particle accelerator at FAIR. It will provide scientists with the ability to explore states of matter in regions of high baryonic densities at moderate temperatures [8] and will allow for the search for rare short-lived particles at unprecedented collision rates of up to 10 MHz [9].

In CBM, heavy ions (e.g., gold ions) will be accelerated to almost the speed of light to collide with a target (e.g., a thin plate of gold), to enforce a nucleus–nucleus collision that creates states of matter with extremely high density [10]. Under these conditions, a state known as Quark–Gluon Plasma can be produced. In this state, quarks and gluons are freed from their usual strong interactions within the plasma. After that phase, during hadronization, quarks combine into new particles that burst out of the central collision point (primary vertex) right into the detector setup.

Some of these newly formed particles decay almost immediately, either due to instability or interaction with other particles. At the point of decay, particles are generally referred to as mother particles, whereas the particles created during the decay are so-called daughter particles. The specific mother particles that decay almost immediately are called short-lived particles, and they can only be measured indirectly by the reconstruction of decays or decay chains, since they rarely reach any detector before. Then, the daughter particles’ trajectories (tracks) are used and extrapolated to find the point of decay (secondary vertex), whereas their properties help to identify the decayed particle.

An efficient search for rare short-lived particles requires the already-mentioned high interaction rates of up to 10 MHz. This creates further computational challenges. At these interaction rates, it is not possible to store the data streams entirely, as they are produced with approximately 1 TB of data per second [11]. To reduce the amount of data that has to be stored, a reduction by approximately three orders of magnitude [12] is required, and can be achieved by selecting only collisions (events) of physicists’ interest. However, in CBM, there is no simple criteria for event selection, as the search for rare short-lived particles requires a full event reconstruction [13], including the reconstruction of decays and decay chains.

An algorithm package called First Level Event Selection (FLES) [14] is used to provide a full event reconstruction in real-time that allows selection of events of interest and, therefore, significantly reduces the amount of data that has to be stored on disk. The algorithms used for the experiment are tested and evaluated step by step using precise Monte Carlo simulated data. Thus, the true outcome of each event is known, and reconstructed particles can be matched with the corresponding simulated particles to allow a performance analysis. The reconstructed particle candidates are identified with a hypothesis, which can be tested and evaluated using simulated data. In real experiments, however, the comparison is not possible, which makes a detailed performance analysis a crucial step for a successful experiment. Therefore, the presented approach will use the already-established tools of the KF Particle Finder to measure the performance within the reconstruction chain of CBM.

The Kalman Filter (KF) Particle Finder is a package included in FLES, responsible for particle and decay reconstruction, including the reconstruction of mother particles. When mother particles are created, the type of particle and their properties are inferred by the properties of the daughter particles. Due to inaccuracies in the reconstruction of daughter particles, the reconstruction of multiple mother particle candidates is not uncommon. For that reason, the particles pass through a particle competition to find the best-fitting candidate. That way, the background noise produced by mistakenly created mother particles can be reduced.

The presented neural network approach is used to replace the existing particle competition of the KF Particle Finder. For each pair of mother particle candidates that would compete against each other, the neural network classifies the best-fitting mother particles based on their properties. The neural network solves the competition by classifying the most probable candidate out of both competitors. Previous work [15] already investigated a multi-layer perceptron-based approach, demonstrating that such models are generally capable of solving the problem with comparable results. The present work shows improvements in the particle competition by using a more complex model, including a different topology and hyper-parameters. The new model provides a raw classification performance with an error of less than 2% on the test set.

2. Materials and Methods

2.1. Particle Competition of $K_s$ and $\mathrm{\Lambda }$

The neutral particles $K_s$-mesons and $\mathrm{\Lambda }$-hyperons serve as important indicators for the CBM experiment. $K_s$ consists of a down-quark and a strange anti-quark, whereas $\mathrm{\Lambda }$ is build with an up-quark, down-quark and a strange quark. Theoretical predictions suggest that enhanced strangeness production (the production of strange, multi-strange or hyper-strange particles, that consist of one, two or three strange quarks or strange anti-quarks) is an indicator of deconfined matter [16], the Quark–Gluon Plasma. Both particles are abundantly created in the energy ranges of the experiment [15] and are therefore a reliable source of information. The two particles decay with the given probability BR (branching ratio) as follows:

$$\begin{array}{cccc}\hfill K_s& \to {\pi }^{+}{\pi }^{-}\hfill & \hfill & (\mathrm{BR}:69.20\pm 0.05\%),\hfill \\ \hfill \mathrm{\Lambda }& \to \mathrm{p}{\pi }^{-}\hfill & \hfill & (\mathrm{BR}:63.9\pm 0.5\%),\hfill \end{array}$$

(please see [17,18], respectively), and therefore both give a rise to negatively charged pions ${\pi }^{-}$. Moreover, all daughter particles of $K_s$ and $\mathrm{\Lambda }$ are non-neutral and, therefore, can be measured by the detectors and reconstructed by the algorithms. The particle reconstruction procedure combines all possible daughter particles, resulting in the simultaneous creation of a “common” pion, which is a decay product for both, the $\mathrm{\Lambda }$-hyperon and $K_s$-meson, leading to the creation of these two possible mother particles, even though only one exist in the Monte Carlo simulated data. As a result, $\mathrm{\Lambda }$ and $K_s$ create background noise for each other, which will hinder the physical analysis of the particles using real experiment data. Performing a particle competition to decide for the best-fitting mother particles helps to reduce this physical background noise by removing the falsely created mother particle candidate.

The Kalman Filter (KF) Particle Finder [19] is an important package included in FLES, responsible for online reconstruction of short-lived particles and their decay chains. The package was updated recently by the Missing Mass method [20], and is now capable to reconstruct more than 150 different particle decays. Furthermore, it includes tools for performance measurements in the particle reconstruction process. The KF Particle Finder package reconstructs all possible mother particle candidates when a decay is recognized. Then, ${\chi }^2$-cuts and a particle competition are applied to select the best-fitting reconstructed particles and to reject the others. The existing implementation of particle competition has several stages. One of the most important steps is the evaluation of particle candidates’ reconstructed mass values. Here, the algorithm examines the reconstructed particles’ masses of each pair of candidates (competitors), and checks if one is within 3$\sigma$ range of the known mass distribution peak, the so-called Particle Data Group (PDG) (which refers to a global collaboration of particle physicists, who define standards and publish results within the scope of research) mass. If one of them is within the range, the mass distance to the respective particle’s PDG mass is used to determine the best-fitting mother particle.

2.2. Data Extraction Using the KF Particle Finder Package

In CBM, the performance of algorithms within the FLES package is measured by a comparison of Monte Carlo simulated data and reconstructed results by the algorithm packages. Using this approach, the reconstruction efficiency and precision of each part of the package can be evaluated. The Monte Carlo data can be considered as the true outcomes of each event, allowing the application of supervised learning techniques. In the stage of the KF Particle Finder, when the particles and decay chains are reconstructed, they are matched with the corresponding Monte Carlo true particle. In this work, the Monte Carlo information of each matched particle is utilized to check if the reconstructed particle is a true $K_s$-meson or $\mathrm{\Lambda }$-hyperon. If this is the case, the information of the corresponding reconstructed particle is extracted to build a dataset. In real experiments, Monte Carlo information does not exist and, therefore, the neural network has to perform on reconstructed information and hypothesis only.

Overall, 25,000 events generated by the UrQMD model [21] were used in this work. A total of 12,000 events were used for training and testing, whereas 13,000 were used to evaluate the network’s performance in comparison to the existing approach. All events are central Au+Au collisions at 10 GeV energy, and therefore within the specifications of the CBM experiment. In general, the created generated particles are processed by a transport engine (e.g., GEANT4 [22]) to simulate the particles flying through the detector system, including all relevant physics processes such as decays, scattering and interaction [23]. At this point, the whole event outcome is simulated, including all decay chains and trajectories. Afterward, the detector responses are generated, resulting in measurements (hits) that would also be produced by real experiment particles when they interact with a detector. The hits are then used to reconstruct tracks and the KF Particle Finder is applied to reconstruct short-lived particles and decays.

The first part of the dataset (12,000 events) is used to train and test the neural network architectures in the Artificial Neural Networks for First Level Event Selection (ANN4FLES) [24] standalone package. Here, a measurement for raw classification performance was obtained by the accuracy and cross entropy loss values per epoch. This dataset was divided by a 80:20 ratio for a training dataset and a testing dataset, respectively. Several fully-connected neural network architectures and settings were tested by hand, where a multi-layer perceptron with three hidden layers with a peak accuracy of more than 98% on the test set was finally chosen.

The second part of that data set, consisting of 13,000 events, is used for comparison of the neural network based approach and the existing particle competition within the KF Particle Finder. Hence, after training and testing, the pre-trained neural networks are evaluated, implemented in the KF Particle Finder package as a part of the reconstruction chain of the CBM experiment. That way, it is possible to use the tools for performance measurement included in the KF Particle Finder, that are a well-established standard to evaluate the algorithms’ performance in heavy-ion physics experiments.

2.3. Performance Measurements in the KF Particle Finder Package

When specific particles are investigated for physics analysis, it is important to achieve clean probes. Due to the large amount of particles (up to 1000) in a collision [25], there are many tracks pointing to the collision point (primary vertex) and, thus, create a large amount of possible track combinations. Due to the limited underlying detector resolution and calculation inaccuracies of floating point representation in a computer system, it is only possible to reconstruct all tracks and particle decays within an acceptable range of imperfection. In several cases, multiple tracks are within a defined error range, such that uncertainties in the reconstruction can not be prevented. Some of the reconstructed particles’ tracks are real tracks that exist, others produce so-called combinatorial background, due to mismatched track segments in the reconstruction process. These can be found by comparison to the simulated data.

When using Monte Carlo simulated data, particles that were reconstructed and classified correctly are called signals, whereas particles that were mistakenly reconstructed are called ghosts, and misclassified particles are generally referred to as physical backgrounds for the respective other particles. The KF Particle Finder package produces histograms for several parameters, such as particle’s mass distribution, and separate histograms for each of these categories. This allows a detailed analysis of the reconstruction performance when working with simulated data.

Beside the histograms, metrics can be calculated based on it. These include the significance and the Signal/Background ratio (S/B ratio). The S/B ratio is just the amount of signal divided by the amount of background, which allows evaluation of the approaches by showing if one or the other approach is rejecting more signal relative to the amount of rejected background. The significance expresses the relation of the signal peak in comparison to (lower) peaks produced by background. A significance of 1 indicates that the background peaks by fluctuation are as large as the signal and, therefore, in a real experiment, the peak would not be recognized as a different particle, as it has no difference to background fluctuations. Contrarily, a significance of 5 is considered a threshold to see a peak as a signal of a particle that should be investigated. However, since $\mathrm{\Lambda }$ and $K_s$ are already well-known and parameters are usually set to find them with a high significance, the threshold for these particles is always achieved, even without competition. Nevertheless, a large reduction in significance should be investigated.

2.4. Training and Testing Using ANN4FLES and PyTorch

The C++ package ANN4FLES is designed for the fast and efficient creation of various neural network architectures and will be optimized for its usage within the full event reconstruction chain of the CBM experiment. In its current state, ANN4FLES includes architectures such as fully connected multi-layer peceptrons, convolutional neural networks, recurrent neural networks, graph neural networks, and more. Furthermore, it provides a graphical user interface for training, testing and fine-tuning of various hyperparameters without the need for additional programming, such that pre-trained neural networks can be easily exported to be used in CBM’s FLES package.

In the present work, a Multi-Layer Perceptron (MLP) [26,27] is used to solve a classification problem between two possible mother particles in a competition based on their reconstructed properties. The ANN4FLES standalone package is used to create the neural networks for training and testing. Then, the chosen pre-trained network is included into the KF Particle Finder package and used in the particle competition to classify the competitors.

Since the previous work [15] already showed that a neural network can perform comparably to the existing competition of the KF Particle Finder, ANN4FLES is used for the first time within the KF Particle Finder package to implement a more complex neural network for this classification task (see Figure 1). ANN4FLES was tested on multiple well-known data sets and it was shown that ANN4FLES offered comparable results to other neural network packages, which makes one confident that the implemented mathematics in ANN4FLES are correct [24]. However, for comparison with a reference network implemented in PyTorch [28], the weights are initialized by the same uniform distribution method [29]. ADAM [30], with a learning rate of $\alpha =0.003$ and default ${\beta }_1,{\beta }_2$, was chosen as a weight optimizer, whereas the selected activation functions are Leaky-ReLU for all hidden neurons and Softmax for the output layer. The loss was calculated using binary cross entropy and the training and testing phase was repeated over 100 epochs with a batch size of 50.

To find a well performing model and settings, different learning rates in the range of 0.001 and 0.005 were tested. The final results are accomplished by a network that was trained with a learning rate of 0.003, whereas lower learning rates performed almost equally, but larger learning rates tend to perform slightly worse. It is assumed that a larger learning rate does not allow for moving as deep into a minimum as with using lower rates, since it might overshoot the minimum slightly.

Besides the learning rates, different layer sizes and network depths were tested. Here, the main focus was set on finding a balance between network size and results. On the one hand, a more complex structure could lead to an even better performance. However, the raw classification performance is already good, such that much deeper networks were not tested yet. A deep neural network increases the amount of parameters and calculations, slowing the classification process and, therefore, has to be balanced with the fast algorithms required for the real-time event reconstruction in CBM. On the other hand, simpler architectures than the chosen one seem to perform worse, which indicates a less complex model is not capable of learning the patterns.

3. Results

Particle Competition Based on Mass and PDG Mass

The MLP implemented using ANN4FLES provides a raw classification accuracy of up to 98.6% for the testing set (see Figure 2). An almost identically constructed neural network was implemented in PyTorch to ensure valid results. The PyTorch architecture provided equally high accuracy values and again confirmed the classification performance of ANN4FLES.

An error of only 1.4% on the testing set suggests that the classification performance of the more complex ANN4FLES architecture is more suitable to solve the task, since the error rate of the previous work is given with more than 10%. There are several possible reasons for the better results, such as the topology, different learning rate or the loss minimizing algorithm, which was Broyden–Fletcher–Goldfarb–Shanno (BFGS) [15,31,32] in the previous work. Furthermore, although the generators for simulated events should perform equally, further investigations are suggested, as previous work used the PHSD [33] model to generate data for the neural network training, testing and evaluation, whereas in the present work, UrQMD is the underlying model to generate event data.

In Figure 3, the total mass spectra histograms for $K_s$ and $\mathrm{\Lambda }$ are shown. The results using KF Particle Finder without competition (black) and with the existing method (green) are visualized for comparison. The ANN4FLES approach is colored red, and one can see that it seems to reduce the number of entries over the whole range. In areas with larger distance to the peak, this is likely to be reduced background. Nevertheless, there is also a reduction in entries at the peak area for $K_s$, where a signal rejection can be the reason. In general, it reduces the amount of particles that were finally classified as $K_s$ or $\mathrm{\Lambda }$, respectively.

Investigating the signal mass histograms in Figure 4 shows that, in both cases, a slight reduction in signal is indicated compared to running the KF Particle Finder without competition. For $K_s$, the ANN4FLES approach rejected slightly more signal compared to the existing method, whereas for $\mathrm{\Lambda }$, the results are the other way around. Here, the existing method rejects more signal particles than ANN4FLES. Thus, one can assume that the largest part of rejected particles in Figure 3 is due to correctly rejected background.

In Figure 5, the background mass distribution is shown. Based on the histogram for $\mathrm{\Lambda }$, it is difficult to see which competition approach is better. Both competitions reduce the background significantly but, around the peak, the existing method seems to perform better, since there is a clearly visible peak for ANN4FLES at $\mathrm{\Lambda }$’s PDG mass $1.116\mathrm{GeV}/c^2$, whereas in the whole range, ANN4FLES seems to reduce the background slightly more. Moreover, there is a large peak of the existing method at $m<1.11\mathrm{GeV}/c^2$, indicating less rejected background by the existing method compared to the neural-network-based approach. For $\mathrm{\Lambda }$, this histogram indicates a tie between ANN4FLES and the existing method. However, evaluating $K_s$, there is no peak at the PDG mass of $0.493\mathrm{GeV}/c^2$ by ANN4FLES. The neural network used seems to perform well in rejecting $K_s$ background around the known mass peak, which, overall, increases the physics analysis for $K_s$. Furthermore, a competition based on the mass is quite difficult, if the reconstructed mass of the background producing particle is within a range where it is expected for the respective investigated particle. These results show that the network is not only classifying by the distance to the mass distribution peak, as is performed in the existing method.

In the ghost histograms (see Figure 6), ANN4FLES seems to be ahead in background rejection in both cases $\mathrm{\Lambda }$ and $K_s$. Although, similar to the existing method, ANN4FLES has a peak around the PDG mass bins, the existing method has more ghosts over the whole range and within the peak area. Thus, the neural network rejects more ghost particles than the default approach and, therefore, helps to reduce the mistakenly created mother particle candidates even more.

In general, the presented plots indicate a better performance of ANN4FLES by rejecting background particles. This can be confirmed by the S/B ratio and significance. In Figure 7, the invariant mass spectra of $K_s={\pi }^{+}{\pi }^{-}$ and $\mathrm{\Lambda }=\mathrm{p}{\pi }^{-}$ for the existing competition are shown. With a significance of 149 and 213 for $K_s$ and $\mathrm{\Lambda }$, respectively, the clear signal is given in both cases using the existing competition of the KF Particle Finder. Additionally, the S/B ratio of 3.58 shows that the data visualized in the $K_s$-plot consists of almost four times more signal than background, where it is over eight times more signal than background for $\mathrm{\Lambda }$.

The following results are achieved by the ANN4FLES approach (Figure 8). Comparing the S/B ratio for $K_s$, one can see that the ANN approach improved the value by around 16%. For $\mathrm{\Lambda }$, however, the S/B ratio has been reduced by about 2%. Considering both particles, ANN4FLES has reduced the background successfully even further. However, for $K_s$, the significance was also decreased by about 3%, whereas the significance of $\mathrm{\Lambda }$ was increased by 1%. That indicates that even if the background was reduced successfully on average, the fluctuation in the background has been increased on average in comparison to the existing method. Nevertheless, both significance values are high enough to consider these particles as a clear signal, making the minor reduction negligible.

4. Conclusions

Summarized, the ANN4FLES-based competition using reconstructed mass and PDG mass can perform comparably to the existing method. In general, using a more complex topology, it reduces background slightly better than the existing approach in the KF Particle Finder, even though the significance is decreased slightly on average. Especially, the amount of ghost particles were reduced almost over the whole range for both particles, and the background reduction around the PDG mass of $K_s$ was strong, even though a similar model might show other results, depending on the learned features. The reason for a good background reduction is most likely that the existing competition is only based on the mass distance to the known PDG mass of a particle, whereas the neural network can also learn patterns between reconstructed mass and PDG mass that can be used to classify particles correctly.

The neural network approach is currently only classifying between $\mathrm{\Lambda }$ and $K_s$, whereas the existing method is furthermore cleaning the background by suppressing, for example, $\gamma$-decays and other cleanup methods. In the presented results, the network was not able to reject a particle entirely, hence neither classifying it as $K_s$ nor $\mathrm{\Lambda }$. The extension of the model by allowing either particle rejection or a classification between more particles that interact as background for each other, might require an increase of the underlying model complexity, but also can help to reduce the background even further.

The ANN4FLES package itself will now be improved with respect to its runtime for the planned applications in the real-time reconstruction chain of the future CBM experiment. After these improvements, ANN4FLES will be integrated into the physics analysis module of the FLES package.