1. Introduction
The main challenge of cognitive science is not only revealing apparent facts of perception but clarifying the mechanisms behind perception and cognition. However, the researchers examining human cognition are always overwhelmed by the complexity of it. For instance, when perceiving a given face, a viewer is able to extract facial identity, expression, and social characteristics (such as attractiveness and competence) accurately and effortlessly [
1,
2,
3].
The modeling of human perception depends on the repertoire of data analysis methods. A certain number of researchers in cognitive science are only equipped with classical statistical analysis tools such as analysis of variance (ANOVA) and linear regression. These researchers are challenged when studying face perception as many of the dimensions in face perception are intertwined. For instance, facial identity (“Who is the person?”) and facial expression (“What is the emotion?”) are widely believed as distinctive. Facial identity is generally regarded as a kind of invariant information that remains consistent within a short period of time, while facial expression is regarded as kind of variant information that is changeable with even tiny muscle movements from time to time [
4,
5]. Conversely, the converging evidence suggested that the perceived emotional expression of a face is affected by the facial identity of that person, thus, these two aspects of a face are inter-dependent with each other [
6,
7]. Similarly, although one may perceive multiple social characteristics from a face, many of these characteristics are heavily correlated with each other [
8]. Several past studies like [
8,
9] even suggested that seemingly complicated social characteristics can be easily represented on a two- or three-dimensional framework. For example, most frameworks believe that the dominance and the trustworthiness of a face are perceived in orthogonal mechanisms.
So far, the researchers in cognitive sciences tend to reduce data dimensions when dealing with face perception data. A common method to conduct dimension-reduction in multiple dimensional data is principal component analysis (PCA), a kind of multivariate method. Briefly speaking, in a typical PCA, the original data matrix (in which data are largely dependent on each other) was transformed into a new matrix formed by the principal components (PCs) calculated via the linear combinations of the original data [
10]. The first PC explains most of the data variance, and the following PCs each explain most of the remaining variance. PCA has been widely used in modeling perception of social characteristics, but there are several concerns regarding only using PCA for this kind of multiple dimensional data.
First of all, PCA is not easy to conduct properly. Though some researchers in face perception have used PCA to reduce data dimensions [
11,
12], the operation procedures of it are not standardized among different labs. In a typical principal component analysis, the original data are supposed to be continuous variables. Whereas, in many human perception studies, the data are measured in discrete Likert scales (e.g., 9 point scale in [
8]). Although many treated such data as a kind of continuous variable, some argued this approximation may jeopardize the rigorousness of data analysis. Furthermore, the implications of principal component analysis require maneuvers that many researchers may not actually understand nor even command. For example, the data rotation operation is always recommended but sometimes left undone [
8]. In a recent large scaled replication study [
13], the authors argued the rotation procedure is vital but has been ignored in some seminal works. Second, PCA is not ideal for all research questions. It operates under the assumption that samples follow some specific distributions, which can lead to meaningless reductions when dealing with data that are not uniformly distributed. For example, real human perception data may contain multiple clusters, but these clusters are not evenly distributed in some unknown dimensions. Furthermore, reducing data dimensions in human perception data is not a necessity. The general logic behind the PCA is to reduce the data dimensions with a mathematical algorithm, but data reduction may not be the omnipotent solution for modeling face perception with a small number of dimensions [
14]. Though computer science researchers utilize PCA as well, the number of the output dimensions (the number of PCs) in their typical studies are much greater than the original number of dimensions dealt with in human cognition studies. For example, in one of the pioneer works using computer vision techniques in face perception [
15], the authors used 50 PCs to reduce the data dimensions of real face images (with
dimensions).
In considering the suitability of PCA as a dimension-reduction technique, t-distributed stochastic neighbor embedding (t-SNE) emerges as an exemplary alternative [
16,
17]. Notably in its distinction from PCA, t-SNE boasts several advantageous properties. As a nonlinear algorithm, t-SNE excels at preserving the local intricacies of data structures [
18,
19]. Tailored specifically for visualizing complex, high-dimensional datasets, t-SNE adeptly generates two- or three-dimensional representations that illuminate data clusters and relationships, which may remain obscured by linear methods such as PCA. Furthermore, the robustness of t-SNE for diverse data characteristics sets it apart. It eschews the assumption of a global Gaussian distribution in favor of a more adaptable probabilistic model, capable of flexibly accommodating a spectrum of data distributions. Furthermore, what is worth mentioning is that t-SNE, a projection-based method [
20], does not cause serious data loss, so its clustering results on the two-dimensional plane can reveal hidden information that PCA may regard as non-principal components, including new resulting clusters, the internal structure of the data, etc.
To validate the reliability of the data-dimension-reduction results, network analysis, a method that takes advantage of data symmetry is necessary [
21,
22,
23]. Network analysis, especially in the fields of cognition and psychology [
24,
25,
26], is a powerful tool for assessing the reliability of clustering results by leveraging the inherent symmetries within the data. By examining the strength of connections within the network, this method reflects the degree of correlation between clustering results, capitalizing on the symmetry of the pairwise clustering correlation data. By identifying areas where these symmetries hold true, network analysis can confirm that the clustering results are not merely a product of random chance but reflect genuine, underlying groupings within the dataset.
It is challenging and unrealistic for scholars who are not majoring in computer science to use complex dimension-reduction tools, even though in some circumstances, reducing the data dimension is unnecessary. In the sense of this, the neural network approach might be the solution. Using state-of-the-art techniques, the researchers in other research disciplines (specifically, computer vision) classified data with more than hundreds of dimensions with neural networks and revealed the inner structure of the data. This fantastic cutting edge technique has been widely utilized in various applications, such as image classification [
27,
28,
29,
30], system identification [
31,
32,
33], natural language processing [
34,
35,
36], autonomous driving [
37,
38,
39,
40] and fault diagnosis [
41,
42,
43,
44]. Thus, the neural networks might be the ideal candidate to further classify perception data. Although some of the researchers are aware of the neural networks, they have difficulties when applying these techniques. Specifically, it may take a long time to learn the programming language and build the environment for the neural network. Thus, it is reasonable to introduce an easy-to-use platform with a graphic user interface (which is close to much of the statistics software that the researchers used) for the researchers in cognitive science who are not familiar with coding.
Considering the aforementioned concerns, this paper offers the multi-dimensional data analysis platform (MuDAP) for researchers in cognitive science. The MuDAP is designed with a standardized pipeline and equipped with state-of-the-art neural network techniques based on existing machine learning libraries in Python. The contribution of this paper is listed as follows:
The framework structure of the multi-dimensional data analysis platform (MuDAP) is introduced.
A graphic visualization data-dimension-reduction algorithm based on t-SNE is utilized to dig the real clusters based on the inner structure of the data.
A network analysis taking advantage of the symmetric structure of the data on correlations between each predicted cluster is performed to verify the reliability of the clustering results based on the result trustworthiness.
An embedded neural network training algorithm is proposed to solve the corresponding regression and classification problems using the cluster results as labels.
A step-by-step illustration of how to use MuDAP, analyzing the introduced face perception experiment data, is shown that verifies the function of the MuDAP.
2. Framework Structure
The multi-dimensional data analysis platform (MuDAP) is built within Python, and its framework structure is first introduced in this section.
2.1. Dataset Import
MuDAP addresses the causal relationship among the class of high dimensional data in cognition science. The imported original data should be obtained from real sconces, such as the decision making or perception collected from a questionnaire survey. In this sense, an element
denotes a set of collected high dimensional data, where
,
is the value at a featured dimension,
m is the total number of element features, and
is the labeled value of that element. All these collected elements give rise to the dataset
and are, hence, stored in the directory ‘MuPAP/LoadDataFile/…’ with the file name ‘data’ in CSV format.
2.2. Graphic User Interface
As shown
Figure A1, MuDAP has a total number of six function buttons in its main welcome screen, and these buttons correspond to the procedures explained below.
1. Dimension Reduction: This function employs the t-SNE method to reduce the high dimensional data to fit a 2D plane and plot all the data with their labels in this plane. It verifies the data structure to confirm if any clusters are formed such that later procedures like regression or classification can be carried out. If so, the users can manually insert the center points of the obtained clusters into the memory.
2. Network Analysis: This function can only be performed after the center points of the clusters are stored. It exploits the relationship between all elements in a cluster and each original featured dimension via network analysis.
3. User Configuration: This function enables the users to tune the training parameters shown in
Figure A2 according to their identical data structure within the later DNN training process.
4. Regression Analysis with DNN: This function trains a deep neural network to predict the distances between any given new element to all existing cluster centers in the 2D plane.
5. Classification Analysis with DNN: This function trains a deep neural network to predict the closest cluster of any given new element in the 2D plane.
6. Contact Information: The developer contact information of MuDAP is shown in this function to allow users to make any direct queries.
The toolbox we have developed operates through a three-step process. Initially, the t-SNE algorithm is utilized to capture the spatial structure of the data without any loss of information. This step is crucial as it provides a comprehensive overview of the data’s inherent dimensions. Second, network analysis is employed to establish direct connections between the various clusters. This analysis is instrumental in validating the relationships within the data and ensuring the reliability of the clustering results. Finally, a neural network is trained to perform regression and classification on the data, which are essential for drawing meaningful insights and predictions. After completing these three steps, the complex and cumbersome high-dimensional cognitive data can be objectively and quantitatively described and analyzed.
2.3. User Instructions for MuDAP
Before conducting any further data analysis, the users must check the data structure of the imported data by using the ‘Dimension Reduction’ function and double check whether the formed clusters have any consistency with the given labels. If there actually exist several clusters, the user can insert the centers into storage and perform the following steps. Then, the ‘Network Analysis’ function is performed to further analyze the relationship between each identical cluster.
After that, the tuning parameters are set in the ‘User Configuration’ function before any neural network training procedures. Hence, the DNN is trained to solve the respective regression and classification problems via the ‘Regression Analysis with DNN’ and ‘Classification Analysis with DNN’ functions. In this sense, MuDAP is capable of discovering the causal relationship between the data structure and data type.
5. Conclusions and Future Work
The researchers in cognitive science have long been interested in modeling and classifying the human perception data. This task requires a powerful and easy-to-use analysis platform. Although classical dimension reduction methods like PCA are powerful for building an initial framework, they are not capable of classifying the data and revealing the inner structure and clustering of these data. DNN, on the other hand, has been well-proven to be an ideal method for this kind of research question. However, DNN is not easy to train and implement. Here, the multi-dimensional data analysis platform (MuDAP) with a graphical user interface has been developed to assist cognitive science researchers in handling complex human perception data and classifying its potential structures. The operations of this toolbox are structured into three steps. Initially, dimension reduction based on the t-SNE algorithm captures the spatial structure of the data, identifying nine cluster centers without information loss. Subsequently, network analysis is employed to establish direct connections between each cluster, thereby verifying the reliability of the aforementioned results. Finally, the nine cluster centers are designated as labels, and a neural network is trained to perform both regression and classification on the data. MuDAP facilitates the objective, qualitative, and quantitative analysis of complex, high-dimensional cognitive data, simplifying the research process for cognitive scientists in fields outside of computer science.
In this paper, analyzing the data from the experiment using MuDAP demonstrated that the platform is capable of elucidating the inner structures of various social characteristics (the DNN) and can show the relationship among the different types of personalities (the network analysis). Moreover, using the MuDAP, we are able to show that the facial characteristics of male faces can be summarized in nine types determined mainly by attractiveness, dominance and masculinity but not expressiveness. This finding is inaccessible following dimensional reduction, which only shows the essential components of the social characteristics. Finally, with the help of our trained DNN, new-coming test data are classified into correct clusters, and their distance to each cluster center is predicted accurately.
The MuDAP is an easy-to-use and powerful data analysis platform for cognitive scientists dealing with multiple dimensional data. With this, researchers without expertise in coding can process massive amounts of data with great ease at the GUI. The output of the data offers data classification, which is useful for multiple dimensional data analysis. Therefore, the analysis from the MuDAP complements the usage of the current data analysis methods.