Hub Traveler Guidance Signage Evaluation via Panoramic Visualization Using Entropy Weight Method and TOPSIS

Abstract

Signage functions as guidance and distribution assistance, directly affecting the operational efficiency of traffic in and around the comprehensive transportation hubs. Among the elements of signage, the visual guidance effect is the key factor affecting the information conveyance, which should be evaluated during the design and optimization process. This study conducted field investigations and developed panoramic videos for multiple transportation hubs in China and designed a survey accordingly. Human subjects were recruited to watch panoramic videos via virtual reality (VR) and respond to the surveys. The results show that the degree of visual attention of travelers significantly affects the evaluation results of guidance signage, with the influence being inversely proportional. Key factors affecting visual attention include accurate legibility, obstruction and defacement rates, informativeness, and whether signage is set up in a hierarchical manner. In unfamiliar environments, travelers focus on the overall context and closely observe the interaction between directional signs and their surroundings. The prominence and visibility of signage are influenced by interactions within the spatial environment. Notably, simple and clear signs are more likely to attract travelers’ attention, and their directional information is more easily comprehended. Moreover, when the destination is clearly defined, visual attention significantly directs pedestrians’ wayfinding behavior.

1. Introduction

In the context of rapid urbanization, the guidance signage system is expanding to meet the growing demand for public travel. In the interiors, exits, and entrances of landmarks and transportation hubs, wayfinding signage plays an important role in guiding traffic flow [1], which also acts as a major source for travelers to obtain destination information [2]. The complex spatial environment inside the hub affects the behavior and information needs of travelers. Whether the guidance signage is complete and accurate or not directly affects travel efficiency [3]. Therefore, it is necessary to carry out a scientific and systematic evaluation of guidance signage from the visual perspective of travelers, as they are the subjects of hub service.

Signage is an indispensable guidance system to assist travelers in integrated hubs. The design and deployment of signage should align with the specific functions, layout configurations, and the unique needs of passengers or pedestrians within transportation hubs. When selecting scenarios for evaluating directional signage, scholars have conducted assessments of guidance signage at various hubs, such as integrated passenger terminals [4], rail transit entry and exit points [5], airports [6], urban roads [7], highways [8], and tunnels [9]. Each of these serves as a representative case study for assessing the effectiveness of internal signage and developing corresponding evaluation frameworks. While the functions, layouts, and types of these hubs differ, the core principles and methodologies for evaluating the effectiveness of internal guidance signage exhibit commonality across different settings.

Surveys were distributed during field investigation and cognitive experiments [10] to establish an evaluation of the guidance signage system in subway stations [3]. When evaluating the guidance signage inside the integrated hubs, the planning and design methods of wayfinding signage were analyzed. Then, the perception and behavioral data of travelers [4] was obtained from the perspective of information needs [11]. After summarizing the setting methods and limitations of signage, the layout method of guidance signage based on traveler behavioral characteristics and spatial positioning model is proposed [12].

When selecting evaluation variables for the objective setting of guidance signage, scholars started with basic elements such as colors, fonts [13], graphics, and quantity [14]. They selected information setting [15], visual recognition [16], etc., to establish the evaluation model of guidance signage effectiveness. Moreover, the criteria and considerations for the installation and evaluation of indoor versus outdoor directional signage vary significantly. Researchers have conceptualized and categorized the factors influencing the effectiveness of indoor digital signage, identifying five primary criteria for evaluation: interactivity, user context, scalability, spatial aesthetics, and visual dynamics.

Furthermore, the integration of eye-tracking devices, VR technology, and driving simulators presents novel methodologies for evaluating signage systems [17]. VR technology facilitates the creation of controlled experimental environments [18], where visual tracking techniques could be employed to analyze eye-tracking metrics, such as gaze duration, for comprehensive visual assessments [19]. By immersing users in computer-generated virtual environments, VR offers a highly engaging and realistic experience. Eye-tracking technology using devices like eye trackers captures detailed data on pupil movement and gaze time by monitoring eye activity. As an objective tool for studying decision-making behavior, eye-tracking has been extensively applied across various domains and has become a valuable method for assessing visual and psychological processes [20,21].

Driving simulators, which offer high safety and repeatability, provide an ideal platform for conducting experimental evaluations of directional signage in environments such as highways and tunnels [22]. Additionally, comparative studies have assessed China’s “public information guidance systems” against international standards, focusing on driver recognition and information prioritization. These studies resulted in the development of an ergonomic model addressing signage information requirements, comprehension, and quantity, which integrates layers of goals, methods, indicators, comprehensive evaluations, and subjective assessments. This model provides a holistic analysis of signage guidance effectiveness from the perspectives of driving behavior, cognitive processing, and system design, culminating in the Airport Roadway Traffic Sign System Evaluation Method (ARTSSEM). These findings underscore the scientific and practical value of simulation experiments in assessing the effectiveness of directional signage systems [23].

The effectiveness evaluation of the guidance signage system is mainly based on subjective investigation and objective experiment, qualitative and quantitative analysis methods. Quantitative evaluation is mainly to analyze the influencing factors and to precisely measure the related variables, for example, visual recognition distance. The reaction time of the subjects when observing the signage was obtained through the eye-tracking instrument, and the amount of information was quantitatively analyzed [24]. Qualitative and quantitative methods are mainly applied to the calculation and analysis of evaluation variables, for example, the order relation analysis method (G1), hierarchical analysis process (AHP), fuzzy comprehensive evaluation method (FCE), and data envelopment analysis (DEA) method [25]. Through modeling, the weights of the indicators are quantified and calculated to evaluate the effectiveness of signage. Among them, G1 and AHP methods are subjective evaluation methods, which are mostly assigned to the indicators through expert ratings. Consequently, these methods are frequently employed in signage evaluation to determine the relative importance of evaluation factors and to validate the effectiveness of signage models.

DEA is a non-parametric objective evaluation method for evaluating multi-input and multi-output indicators. The Entropy Weight Method (EWM) proposes the concept of information entropy. The smaller the information entropy is, the larger the indicator weight is, which could fully reflect the objectivity of the weights. The Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method ranks the proximity of the evaluation object to the idealized target, which has a good application effect in multi-objective planning problems [26]. Quantitative calculation methods are highly objective and avoid the shortcomings of traditional methods that require subjective empowerment. In summary, the evaluation of directional signage effectiveness typically integrates both subjective and objective approaches. By designing relevant evaluation indicators and selecting appropriate evaluation models, these methods provide a comprehensive assessment of factors such as visual clarity and guiding effectiveness of signage.

Overall, the research on the evaluation of traffic signage at urban intersections, station entrances, and exits has been relatively completed. The main contents include traffic management and road safety facility system evaluation. The focus is around visual evaluation indicators such as signage setup, visual information, and effects. Because of technical limitations, surveys and on-site interviews are the main methods for evaluating the guidance signage in the comprehensive hubs. Integrated hubs need to realize the collection and distribution of traveler flows within a short period. Considering the limitation of travel time, some travelers may have a low willingness to fill in the survey. This may lead to bias, and reduce the recovery rate and validity of the surveys. AHP and FCE are the main subjective evaluation methods. These methods could introduce variability in respondent selection, as individuals who are not familiar with the specific hub may be included. This lack of familiarity could compromise the objectivity of the evaluation results, potentially resulting in lower response rates and reduced survey efficiency. Evaluation criteria primarily address the visibility of signage, the clarity of information provided, and its overall effectiveness.

Based on the application of eye-tracking and VR technology, quantitative metrics such as gaze time and reaction time are considered. With the application of eye-tracking devices and virtual reality (VR) technology, quantitative metrics such as gaze duration and reaction time could now be used to assess signage performance. Within this literature, it has been shown that, although caution is needed, VR could be used for pedestrian behavior research [27]. Consequently, scholars have proposed an evaluation framework based on VR technology. This approach first integrates Building Information Modeling (BIM) with virtual reality and, combined with pedestrian or passenger wayfinding behaviors inside buildings, offers a method for evaluating and optimizing signage effectiveness [28]. Participants wear specialized equipment, and metrics such as wayfinding time and wrong turns are recorded, with wayfinding theory being used to describe pedestrian behaviors. Finally, the TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) method is employed to quantify the effectiveness of information signage in guiding passengers.

Furthermore, experimental designs utilizing panoramic video may effectively address this challenge. Panoramic video is an all-round spherical video [18] with realism and immersion, which makes the viewers feel as if they were there [29]. Using panoramic video could highly reproduce the real scene and allow sufficient time for human subjects. Experiments conducted with panoramic video allow participants to interact with realistic scenarios in comprehensive transportation hubs, thereby yielding more objective evaluations. Subjective evaluation methods, such as the Analytic Hierarchy Process (AHP), incorporate significant qualitative elements and are therefore susceptible to subjectivity. The effectiveness of wayfinding signage is influenced by numerous factors, leading to a diverse set of evaluation criteria. As the number of indicators increases, methods like AHP are used to generate extensive data, complicating the assignment of subjective weights to these indicators. Consequently, it is crucial that respondents are well-acquainted with the transportation hub to ensure the accuracy and reliability of the evaluation results. Additionally, objective and quantitative methods should be employed for indicator weighting and calculations.

Travelers are engaged in a “perception-cognition-behavior” process as they navigate through signage in the hubs. Visual attention is purpose-orientated and is the first step to identifying direction. By selecting relevant information within the field of view and making it appear in the center of vision, the human visual system is better able to perceive information details [30]. At this point, travelers’ visual attention and mental activities are directed and focused on the information conveyed by the signage. This shows the importance of signage visuals in directing the visual attention of travelers and improving wayfinding efficiency. However, in the existing evaluation schemes, most of them are combined with scenarios such as traveler wayfinding behavior [14], pedestrian crossing [31], and autonomous driving [32]. The evaluation scheme based on visual attention could really design and optimize the signage from the perspective of travelers’ visual needs. Additionally, the layout of effective signage is critical for traffic safety. Previous research has largely concentrated on the visual behavior of traffic participants regarding individual signs. By employing eye-tracking technology to gather eye movement data, researchers have developed methods for quantifying the information load of signage groups and visual cognition models. These studies have established a comprehensive evaluation method for signage visual cognition in tunnels, thereby corroborating the reliability of the visual experimental methods proposed in this paper [33].

A review of the literature reveals that current approaches to evaluating directional signage primarily rely on static images or simulators. While these methods could display specific scenarios, they lack the immersive and realistic qualities of panoramic video, making it difficult to replicate participants’ actual reactions in real-world environments. Additionally, simplistic scenario presentations fail to capture how signage performs in complex conditions, such as crowded or intersecting pedestrian flows, where cognitive and behavioral responses are significantly affected. Static images or 3D models used for signage evaluation do not adequately convey spatial relationships and layout impacts, overlooking the dynamic nature of real-world environments. Field studies, though useful, often suffer from low data collection efficiency and high associated costs. In contrast, panoramic video not only better simulates complex environments and offers a sense of spatial awareness, but also effectively captures how signage performs across various scenarios, helping to determine whether its effectiveness is influenced by surrounding environmental factors. Overall, the use of panoramic video in evaluation offers notable advantages in terms of realism, dynamism, and efficient data collection. It also addresses several limitations inherent in traditional methods, providing a more comprehensive and accurate analysis of directional signage effectiveness.

This study introduces an evaluation approach centered around panoramic video and traveler visual attention, thus exploring the interaction between pedestrian visual attention and signage evaluation. Initially, through virtual reality (VR) experiments and surveys, a comprehensive preliminary investigation was conducted at various transportation hubs. This approach focused on pedestrian visual characteristics to identify quantitative indicators for different categories and configurations of directional signage. The subsequent evaluation experiment design employed panoramic video and eye-tracking technology in conjunction with surveys to assess pedestrian visual attention to various signage effects.

Furthermore, a panoramic video-based evaluation experiment was designed to assess pedestrian guidance signage. To investigate the relationship between pedestrian familiarity with the hub and evaluation outcomes, both baseline and comparative experiments were conducted. Based on survey results, an evaluation model for signage effectiveness was developed using the entropy-weight method and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). This model analyzed pedestrian visual focus and attention to signage, revealing the intrinsic relationship between signage visual effectiveness and evaluation outcomes through quantitative analysis of pedestrian evaluations and visual attention. The experiments and evaluation methods proposed offer a novel and practical approach to optimizing signage in integrated transportation hubs. Additionally, a multi-task comparative experiment was carried out with case studies of Pudong Airport and Xi’an North Station to analyze the performance of different signage configurations in complex transportation hubs. This study examined the applicability and differences of directional signage across various hubs, providing valuable insights for scenario-based signage optimization.

Thus, based on a thorough literature review and research summary, this paper primarily addresses the following critical issues:

• Impact of Visual Characteristics on Pedestrian Evaluation: How do visual characteristics of directional signage (such as category, information volume, and configuration) influence pedestrian visual evaluation results within transportation hubs? The study integrates pedestrian visual attention to signage, constructing an evaluation method that assesses visual guidance effectiveness from three perspectives: accuracy of visual recognition, contextual placement, and clarity of information delivery.
• Integration of Traditional and Advanced Evaluation Methods: How could conventional survey methods be integrated with panoramic video, VR testing, and eye-tracking technologies to propose a streamlined, practical, and scientifically rigorous approach for evaluating signage effectiveness?
• Experimental Design for Complex Transportation Hubs: Given the diversity and complexity of transportation hubs in terms of size and function, how could multi-task comparative experiments be structured to evaluate different signage configurations? Furthermore, how might the proposed evaluation methods be validated to ensure their reliability and applicability across varying hub environments?

2. Methodology

The comprehensive evaluation of guidance signage is a complex and systematic endeavor. Evaluation methods based on field experiments and pure surveys are easily constrained by real-time conditions. This leads to problems such as external interference factors and low survey response rates, resulting in less legitimate evaluation. The comprehensive assessment of guidance signage is a complex and systematic undertaking. Methods for evaluation, relying solely on field experiments and surveys, are often hindered by real-time constraints. This gives rise to challenges such as external interference factors and low survey response rates, ultimately compromising the legitimacy of the evaluation.

2.1. Property Experiment Overview

In this study, a multi-task and multi-scene experiment utilizing panoramic video was adopted, building on previous research methodologies and informed by an extensive field investigation. The research design was guided by well-established approaches to visual attention and human factors in signage evaluation, integrating panoramic video to replicate realistic scenarios. We carefully selected evaluation variables that take into account the visual characteristics of travelers and introduced a novel evaluation approach for the guidance signage around the hub. Panoramic video was employed as the primary data source, enabling the creation of multi-task comparative experiments designed to reflect real-world conditions in integrated transportation hubs. This approach was selected due to its ability to simulate complex environments and capture pedestrian interactions with signage in a controlled manner. The evaluation framework integrates EWM and TOPSIS, both chosen for their effectiveness in multi-criteria decision-making and their proven application in similar signage evaluation studies.

The next section provides a detailed description of the research methodology.

Within comprehensive transportation hubs, the guidance signage is meticulously arranged, and the pathways for traveler flow are varied and abundant. Following field investigations, Pudong Airport was chosen for a preliminary experiment to assess the reliability of the proposed approach. Surveys, virtual reality (VR), and photo/video surveys were employed to capture travelers’ visual evaluations of the signage. Ultimately, visual evaluation variables were refined, encompassing visual recognizability, setting situation, information expression, and visual attention.

The formal experiment comprises both a panoramic video experiment and surveys. Prior to viewing the panoramic video, a pre-experiment survey was administered to collect demographic information and social attributes of the human subjects. Throughout the experiment, participants watched the panoramic videos while equipped with an eye-tracker. Following the experiment, a post-experiment survey was conducted to capture the written thoughts of the human subjects

The proposed method should be validated for generalizability across various settings. In this study, Shanghai Pudong Airport and Xi’an North Railway Station were chosen for comparative analysis. Both large-scale comprehensive transportation hubs share similar passenger flow patterns. Additionally, their internal guidance signage systems are diverse in both type and function, making them ideal case studies for validation. The experimental design was validated using the EWM and TOPSIS methods. The study compared the significance of various evaluation indicators in integrated hubs across different cities. Through a comprehensive comparative analysis, the research delved into the strengths and limitations of the signage design, ultimately proposing targeted improvement strategies (Figure 1).

2.2. Preliminary Experiment

The primary objective of the preliminary experiment was to assess the feasibility of the panoramic video-based experiment. A survey was undertaken to gather travelers’ evaluations of the visual characteristics of the signage and to refine the evaluation variables.

Field Investigation

Shanghai Pudong Airport was selected as the location for the preliminary experiment. Inquiry surveys, VR video surveys, and image/video surveys were utilized for research purposes. By comparing the outcomes obtained through different survey methods, the presence of consistency would suggest the feasibility of the experimental program.

Surveys were distributed both online and in paper format at Pudong Airport. The survey covered respondents’ basic information, their familiarity with and frequency of using the integrated hub, as well as their evaluations of the guidance signage.

The survey was constructed using a set of evaluation indicators derived from the design principles of the guidance signage system, encompassing conspicuity, discernibility, rationality, continuity, and integrity. Consequently, in the survey design, the visual characteristics of travelers and the objective setting of the signage were amalgamated. The chosen evaluation variables cover facets of objective visual recognition, signage setting, and information expression. The detailed connotation of each indicator is further expounded as follows:

• Visual Recognition: Visual Recognition serves as the cornerstone for travelers to gather information for wayfinding behavior, ensuring that travelers can precisely capture visual information at first glance. From the perspective of intuitive visual effect, four indicators such as accurate readability, size, brightness, obstruction, and defacement rate have been specifically chosen.
• Objective Setting: Appropriately positioned signage captures travelers’ attention effectively. The strategic placement of signage directs visual attention in a focused manner, allowing travelers to promptly and accurately capture directional information. Therefore, three indicators such as the location and arrangement of signage, as well as the eye-catching quality of information, have been selected.
• Information Expression: Signage conveying precise information enhances travelers’ visual perception and streamlines information processing. In addition to meeting the criteria of being concise and clear, and providing accurate guidance, signage should also ensure a clear and consistent hierarchy. A judicious amount of information facilitates travelers in rapidly accessing effective information, thereby improving the efficiency of guidance. Therefore, the five indicators are concise clarity, guidance accuracy, information hierarchy, coherence consistency, and amount of information.

The VR survey involves converting the captured panoramic video into a VR video and inviting volunteers to wear VR glasses for an immersive experience and evaluation. The video includes the guidance signs on the route from the arrival exit to each transportation connection station. Picture/video survey is chosen as the control group, and volunteers are invited to watch the actual video and pictures of the hub and evaluate them. Then, the evaluation results from the questionnaire survey are compared with those of the VR survey to confirm the reliability of the VR results (Figure 2).

A total of 65 responses were collected from the field investigation. Among the participants, 55.38% were male and 44.62% were female, reflecting a nearly balanced gender ratio. Concerning age, over 90% of the passengers fall within the 18 to 50-year age range. The preliminary survey was conducted at Shanghai Pudong International Airport, where the selection of both age and gender categories demonstrated a degree of randomness. Consequently, the distributions of gender and age were deemed to be reasonably representative.

Beyond gathering demographic data, the survey incorporated a five-point Likert scale tailored to the previously outlined criteria for evaluating directional signage. To ensure the robustness of this preliminary scale, rigorous statistical analyses were conducted to evaluate its reliability and validity. This process involved comprehensive reliability testing and validity assessment to ascertain that the scale effectively measures the intended constructs related to signage evaluation.

Therefore, a reliability analysis was performed on the collected data during the pilot study following the design and administration of the survey. The analysis yielded a reliability characterization coefficient of 0.862, demonstrating a high level of internal consistency among the evaluation variables. This coefficient suggests that the metrics used for assessing signage performance were highly correlated and accurately reflected the intended evaluation framework. Consequently, this reliability measure provides a robust foundation for the credibility of the study’s findings and serves as a reference point for ensuring the reliability of future experimental designs and survey instruments.

Moreover, to assess the construct validity of the survey, exploratory factor analysis (EFA) was performed. The analysis yielded a Kaiser-Meyer-Olkin (KMO) value of 0.867 (p-value < 0.001), confirming the suitability of the data for factor analysis and indicating high validity. The EFA identified key factors related to signage evaluation, such as accurate readability, size, brightness, obstruction, defacement rate, location, signage arrangement, and the eye-catching quality of the information. Additionally, higher-order factors like concise clarity, guidance accuracy, information hierarchy, coherence consistency, and amount of information were also extracted. These factors explain the underlying structure of the signage evaluation metrics, ensuring that the questionnaire effectively captures the critical dimensions of pedestrian visual assessment within transportation hubs. The ICC (Intraclass Correlation Coefficient) was utilized for assessment to determine whether the findings of the same group of respondents were correlated. The mean measurement correlation coefficient ICC = 0.862 > 0.75 (p-value < 0.001).

Following the reliability, validity, and correlation tests, the survey data exhibited strong internal reliability and validity, confirming the scientific and effective nature of the data collection method. The evaluation results remained consistent even when employing different study methodologies. The method based on panoramic video was found to be feasible when compared to survey and field investigation approaches.

In summary, to ensure the reliability and validity of the survey scale, we initially employed EFA and reliability characterization coefficients. These methods were essential for refining the evaluation indicators used in our study. EFA identified the underlying factors of the evaluation criteria, confirming that the metrics for assessing directional signage were both reliable and valid. The results of these preliminary analyses demonstrated that the proposed evaluation indicators were effective for assessing signage within transportation hubs, thereby establishing a robust foundation for the subsequent experimental design and survey administration.

With the validated and reliable indicators, we could apply the EWM to assign appropriate weights to each evaluation criterion. EWM utilized the validated factors from EFA to ascertain the relative importance of each criterion. Subsequently, the TOPSIS was employed to rank and assess the effectiveness of signage designs within the hubs, as well as across different hubs. By incorporating the weights derived from EWM, TOPSIS facilitated a comprehensive and objective analysis, allowing for the prioritization of signage options based on their performance.

2.3. Formal Experiment via Panoramic Video

2.3.1. Experiment Environment Settings

The panoramic video was captured using a binocular camera. To simulate a wide range of human perspectives and create an immersive experimental scene, the video was presented on a triple-screen monitor. The eye tracker utilized two sets of cameras: one for eye tracking and another for capturing the scene (Figure 3).

2.3.2. Human Factors

Human subjects were recruited randomly, with a predominant representation of students and staff members due to the experiment being conducted in a university setting. To ensure the accuracy and validity of the experimental data, participants were required to be in good health. Regular vision was also a prerequisite to prevent potential issues with eye-tracking, which could result in blank or inaccurate visual data.

The scenarios were categorized into experimental and control groups, each undergoing two distinct experiments: taskless and task-driven. Each scenario’s experiment lasted approximately 8–10 min. The two types of experiments were conducted as follows:

• Taskless Experiment: Participants were allowed to watch the experimental video initially, solely to become familiar with the scene. The purpose of the experiment was not disclosed to the subjects. They were instructed to observe the experimental scene to simulate the typical attention state of travelers to signage in a general context.
• Task-Driven Experiment: Participants were directed to pay attention to specific signs during the video viewing process and were tasked with evaluating the guidance system. Subjects were required to navigate toward a designated destination to explore the guiding role of signage in the traveler’s wayfinding process.

2.3.3. Questionnaire Design

The survey underwent adjustments based on the results of the preliminary experiment. Additional investigations, such as visual attention and signage perception, were incorporated.

To further refine the evaluation indicators, the results from the field research were integrated. New indicators were introduced to directly characterize visual attention. Travelers demonstrate varying degrees of visual attention to signage during wayfinding behavior and when not actively engaged. Therefore, from the subjective perspective of travelers, indicators measuring the degree of visual attention were proposed.

Recognizing that travelers do not allocate equal attention to each sign when relying on wayfinding signage, indicators reflecting the attention degree to individual signs as well as the average attention were introduced. The enhanced evaluation system for wayfinding signage now includes objective visual recognizability, signage setting situation, information expression, and visual attention (Figure 4). The questionnaire utilized in this study was exclusively intended for experimental research purposes, with participants’ personal information maintained in strict confidentiality. Participants have the freedom to withdraw from the study at any stage of the process.

Before the taskless and task-driven experiment, the questionnaire was used to gather the biographic information, social attributes, and signage cognition of the subjects, and consists of two parts: personal situation includes the purpose of hub use, wayfinding, and other questions; and signage cognition was investigated by selecting guiding, locating, and warning functional signs (Figure 5).

Before the taskless and task-driven experiments, a questionnaire was employed to collect biographical information, social attributes, and signage cognition of the subjects. The questionnaire comprised two parts:

• Personal Situation: This section gathered information on the subjects’ biographic details and social attributes, and included questions about the purpose of hub use, wayfinding habits, and other relevant inquiries.
• Signage Cognition: The second part of the questionnaire focused on investigating signage cognition. Subjects were asked to express their perceptions by selecting guiding, locating, and warning functional signs, providing insights into their understanding and interpretation of these signage elements (Figure 5).

Both the taskless and task-driven experiments were conducted to collect subjects’ evaluations of signage in four key areas: visual recognizability, setup, message presentation, and visual attention. Specific attention was given to assessing single-sign attention and average per-sign attention, which were not directly derived from the survey. In the survey design, the fuzzy calculation of the index was implemented by selecting specific signage design issues. This approach allows for the elimination of guidance signage with redundant deployment or insufficiently complete additions based on the identified indicators (Figure 6).

2.3.4. Experiment Scenarios

Traveler flow lines in large comprehensive hubs primarily encompass inbound transportation modes such as high-speed rail and airplanes, as well as outbound transfer options like buses and cabs. Guidance signage plays a crucial role in these hubs, providing travelers with orientation information through transfer guidance and district guidance.

Following a comprehensive comparison of scenarios, Shanghai Pudong Airport was chosen as the experimental group, with Xi’an North Railway Station serving as the control group for the purpose of comparing and verifying the experimental method. Pudong Airport, serving as one of China’s gateway hubs, is equipped with diverse functions and intricate signage, including both conventional signs and unique ones like magnetic levitation and rail transit. This selection allows for a comprehensive evaluation of different signage types with distinct characteristics.

Similarly, Xi’an North Railway Station, acting as a comprehensive city hub with substantial traveler flow, is connected to Xianyang Airport. In addition to signage for railroad services, it features signs for the airport intercity high-speed rail and other informational signs. When capturing panoramic video, the focus was on covering the core functions and typical signs of the hub. An example of the panoramic video is provided here (Figure 7).

2.3.5. Experiment Procedure

Here is a detailed breakdown of the experimental procedure:

• Step 1: Set up the video playback equipment and the eye-tracker, confirming that the pupil data was accurately collected. Introduce the experimental procedure and ensure subjects wear the eye-tracker. Then adjust the subjects’ posture and distance from the screen.
• Step 2: Prior to watching the video, fill in the pre-experiment survey questionnaire to investigate the subjects’ basic attributes and signage recognition (Figure 8).
• Step 3: Subjects watch a panoramic video for a taskless type of experiment, then fill out the evaluation questionnaire for the first time after the video was viewed.
• Step 4: Subjects watch a panoramic video for a task-driven experiment, then fill out the post-experimental questionnaire at the conclusion (Figure 9).

This structured approach ensures the systematic execution of the experimental process, from set-up to data collection and survey completion at different stages of the experiment.

2.4. Data Analysis

At the conclusion of the experiment, both survey and eye movement data were collected for subsequent analysis and evaluation using the Entropy Weight Method (EWM) and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS).

In 1948, Shannon introduced the concept of information entropy, which characterizes the information contained in random events for the first time in information theory [34], and it is now widely used in the fields of economics, engineering, etc. [8,35]. EWM determines the weights of evaluation indicators by analyzing information entropy and the degree of variation among these indicators. Greater variation among indicators results in higher weights being assigned to those with more substantial information content.

In this experiment, the EWM followed a general process. The original data matrix $A={{(a}_{ij})}_{m\times n}$ was constructed based on the evaluation data and then standardized to $B={{(b}_{ij})}_{m\times n}$. Following the calculation of the entropy weight w_j, the weighting matrix was computed as $c_{ij}=b_{ij}\ast w_j$.

Consequently, this method effectively elucidates the contribution of various characteristics of directional signage to the evaluation outcomes. By mitigating the influence of subjective weighting, it enhances the assessment of directional signage effectiveness across diverse transportation hubs.

The survey was meticulously designed around the variables of objective visual recognition, signage setting, information expression, and visual attention. Likert’s five-level scale method was employed for question setting, and a linear weighting method was utilized to rank each evaluation, facilitating a comprehensive assessment of the effectiveness of guidance signage in different hubs.

TOPSIS was subsequently applied to compare the importance and order of importance of different evaluation variables in distinct cities. It was proposed by Hwang et al. in 1981 and could be used to solve multi-object decision-making and evaluation problems [36]. TOPSIS proves highly effective in synthesizing multiple evaluation criteria for directional signage, including information content and visibility, thereby offering a precise assessment of signage performance.

The fundamental idea involves calculating the distance ($d_i^{\mathrm{*}}$, $d_i^0$) of each evaluated object to the positive and negative ideal solutions. The relative closeness ($f_i$), derived from these calculations, signifies the proximity of the evaluated object to the ideal target.

$$f_i={\displaystyle \frac{d_i^0}{d_i^0+d_i^{\mathrm{*}}}}$$

(1)

Upon sorting $f_i$ from small to large, the relative importance ranking of each evaluation index was obtained. Further calculations based on the standardized matrix of EWM, including the determination of positive and negative ideal solutions and relative closeness for each evaluation index, were performed to complete the importance ranking of each evaluation index.

Given the simplicity and efficiency of the Entropy Weight Method and TOPSIS, the methods have garnered substantial attention from researchers addressing multi-criteria decision-making problems. Compared to methods such as AHP and fuzzy comprehensive evaluation, the approaches employed in this study offer enhanced objectivity in the evaluation process. Furthermore, they facilitate the integration of multi-dimensional data for weighting and ranking, thereby providing a more scientifically robust and comprehensive assessment.

3. Result

3.1. Experiment Results

A total of 36 participants completed the preliminary experimental survey. The correct rates for the cognitive survey of guidance, positioning, and warning-type functional signs were 94.44%, 91.67%, and 94.44%, respectively, indicating that the subjects were generally proficient in recognizing the guidance signage (

Table 1. Basic Attributes of Subjects.

Item	Variable	Percentage (%)		Item	Variable	Percentage (%)
		Test1	Test2			Test1	Test2
Gender	Male	80.77	70.97	Education	Secondary and lower	3.85	0
	Female	19.23	29.03		Post-secondary	96.15	100
Age	<20	11.54	12.9	Transport related	Yes	57.69	61.29
	20–35	84.62	80.65
	35–50	0	3.33		No	42.31	38.71
	≥50	3.85	3.33
Frequency	>6	0	0	Frequency	1–3	26.92	6.45
	3–6	7.69	0		0	63.58	93.55
Purpose	Business	7.69	0	Wayfinding mode	Signage	84.62	93.55
	Traveling	7.69	12.9		Navigation	50	61.29
	Visiting	11.54	6.45		Inquiry	3.85	12.90
	Pickup	3.85	3.23		Signage and navigation	42.31	58.06
	Others	73.08	77.42		Others	0	0

).

In the experimental scenarios at Pudong Airport (Test1) and Xi’an North Railway Station (Test2), 26 and 31 human subjects participated, respectively. The analysis of basic attributes revealed correlations with the degree of cognition. However, the coefficient characterizing the significance of the correlation, with a p-value > 0.05, was not statistically significant.

3.1.1. Eye-Tracker Results

An eye-tracker recorded subjects’ eye movements, capturing data like pupil data collection length and mean fixation duration. Pupil data acquisition duration consistently aligned with the video duration, serving as a metric for subjects’ attentional focus (Figure 10).

The first experiment had a longer duration than the second. Subjects’ visual attention varied based on the experimental task, focusing on the overall environment in one and the guidance sign in the other. This highlights the dynamic nature of visual attention during the evaluation process, emphasizing the importance of an evaluation scheme that considers travelers’ visual attention.

The first experiment lasted longer than the second. Subjects’ visual attention shifted between the overall environment and the guidance sign based on the task. This underscores the dynamic nature of visual attention during evaluation, highlighting the need for an evaluation scheme that considers travelers’ visual focus.

3.1.2. Questionnaire Survey Results

Subjects completed the evaluation survey at the experiment’s conclusion. In an unfamiliar environment, travelers focused on the overall environment, particularly the relationship between guidance signage and the surroundings. The visual attention survey results indicate a noticeable change in travelers’ attention levels. The tables below display the results of the taskless experiment evaluation (

Table 2. Evaluation Results of Pudong in Taskless Experiment.

	Not at All	Slightly	Moderately	Very	Extremely
Readability	0%	0%	12%	35%	54%
Blocking rate	0%	4%	4%	12%	81%
Completeness	0%	4%	23%	54%	19%
Concise clarity	0%	0%	12%	62%	27%
Hierarchy	0%	4%	31%	31%	35%
Consistency	0%	4%	19%	50%	27%
Informativeness	4%	0%	4%	8%	85%
Visual attention	0%	4%	38%	31%	27%
Attention of sign	12%	58%	0%	31%	0%

and

Table 3. Evaluation Results of Xi’an in Taskless Experiment.

	Not at All	Slightly	Moderately	Very	Extremely
Readability	0%	0%	19%	45%	35%
Blocking rate	0%	6%	3%	55%	35%
Completeness	0%	3%	26%	58%	13%
Concise clarity	0%	13%	26%	48%	13%
Hierarchy	0%	19%	26%	52%	3%
Consistency	0%	0%	10%	84%	6%
Informativeness	16%	0%	29%	3%	52%
Visual attention	0%	13%	45%	23%	19%
Attention of sign	3%	16%	0%	55%	26%

).

In the task-driven experiment, participants were asked to navigate within the hub, searching for destinations and signage along pedestrian pathways. This suggests that participants may focus more on the relationship between the overall environment of the hub and the directional signage. Consequently, their evaluation results shifted, reflecting changes in how they perceived the signage itself. These changes, influenced by varying levels of visual attention, were primarily evident in the evaluation of signage visibility and information clarity. The tables below show the evaluation result of the task-driven experiment (

Table 4. Evaluation Results of Pudong in Task-driven Experiment.

	Not at All	Slightly	Moderately	Very	Extremely
Readability	0%	0%	0%	50%	50%
Blocking rate	0%	8%	0%	8%	85%
Size rationality	0%	0%	0%	69%	31%
Brightness	0%	0%	8%	65%	27%
Conspicuity	0%	0%	12%	54%	35%
Concise clarity	0%	0%	8%	65%	27%
Informativeness	0%	0%	0%	8%	92%
Visual attention	0%	0%	8%	19%	73%
Signage attention	42%	42%	0%	15%	0%
Average attention	4%	35%	50%	12%	0%

and

Table 5. Evaluation Results of Xi’an in Task-driven Experiment.

	Not at All	Slightly	Moderately	Very	Extremely
Readability	0%	0%	6%	58%	35%
Blocking rate	0%	0%	6%	32%	61%
Size rationality	0%	16%	13%	61%	10%
Brightness	0%	6%	26%	48%	19%
Conspicuity	0%	13%	13%	48%	26%
Concise clarity	0%	16%	32%	39%	13%
Informativeness	26%	0%	42%	3%	29%
Visual attention	0%	0%	19%	23%	58%
Signage attention	19%	32%	0%	42%	6%
Average attention	19%	32%	0%	42%	6%

).

In legibility, defacement rate, signage size, and brightness surveys, Pudong Airport outperforms in signage visibility. When subjects focus on the signage, their attention primarily goes to objective visual recognition and the signage’s objective setting. This suggests some signs at Xi’an North Railway Station lack visibility and have relatively unreasonable sizes.

Conversely, Xi’an North Railway Station’s signage is more eye-catching than Pudong Airport’s. However, when focusing on guidance signage amidst other signs and the overall environment, it falls short in prominence and eye-catching effectiveness. As a comprehensive terminal, Pudong Airport, with its long walking distances, requires more extensive signage. The scarcity of signage leads to inefficiencies in travelers’ information access, resulting in insufficient evaluation of eye-catching aspects.

The evaluation of information quantity and coherence for single signs at Xi’an North Railway Station is subpar, contributing to low efficiency in travelers obtaining information. Redundancies in information settings within the hub lead to inadequate guidance efficiency. Conversely, positive evaluations of signage information expression at Pudong Airport are higher, with an appropriate amount of information in a single sign. In the overall environment, a simple and clear sign is more easily captured by travelers. When focusing on the signage itself, travelers prioritize direct access to effective and concise orientation information.

Compared to the first experiment, the fuzzy evaluation index of visual attention exhibits a wide range of variation. Subjects’ attention to signage is insufficient when observing the entire experimental scene. However, when the destination is clear, travelers’ visual focus intensifies on the signage, significantly increasing their attention to it.

3.2. Experiment Result Interpretations

3.2.1. EWM Results

The initial survey data were uniformly weighted using the entropy weighting method to derive the evaluation score for each scenario. Given that the survey designs consist of positive questions, higher variable scores indicate better evaluations. By comparing the evaluation results of the four experiments, the indicator weights are determined through the EWM (

Table 6. Weighted Scores and Weights.

	Pudong 1	Pudong 2	Xi’an 1	Xi’an 2	Weight
Readability	0.061	0.083	0.067	0.017	0.036
Blocking rate	0.026	0.018	0.029	0.019	0.078
Size rationality	0.041	0.087	0.027	0.025	0.095
Brightness	0.023	0.020	0.029	0.017	0.058
Location rationality	0.076	0.010	0.021	0.012	0.110
Completeness	0.031	0.018	0.023	0.010	0.044
Conspicuity	0.057	0.027	0.020	0.025	0.080
Concise clarity	0.052	0.020	0.046	0.027	0.051
Guidance accuracy	0.026	0.027	0.054	0.011	0.051
Hierarchy	0.041	0.024	0.052	0.033	0.046
Consistency	0.032	0.026	0.025	0.063	0.109
Informativeness	0.022	0.018	0.064	0.040	0.068
Visual attention	0.041	0.022	0.051	0.038	0.039
Signage attention	0.068	0.049	0.029	0.033	0.066
Average attention	0.059	0.020	0.034	0.024	0.070

).

Based on the weighted scores and their corresponding weights, further comparisons were conducted using linear weighting. The results are as follows: in the first experiment, Pudong Airport scored 0.648, while Xi’an North Station scored 0.479; and in the second experiment, the scores for the two scenarios were 0.228 and 0.219, respectively. It is crucial to note that these calculated results are employed solely for relative comparisons of guide signage across different hubs. Different evaluation metrics may highlight distinct characteristics of the signage. The weights of the evaluation indicators reveal the strengths and weaknesses of the signage across various attributes.

In general, the signage evaluation results for Pudong Airport, i.e., signage effectiveness, outperformed those of Xi’an North Station in both experiments. As the experimental group, Pudong Airport consistently scored higher than Xi’an North Station in terms of signage size reasonableness, eye-catchiness, and the amount of information. This alignment with actual scenarios reinforces the superior signage effect of Pudong Airport (Figure 11).

As a control group, the signage system at Xi’an North Railway Station received an overall positive rating, particularly in terms of visual recognizability and information. Travelers reported a good intuitive visual experience and accurate acquisition of signage guidance information. However, the evaluation of signage information expression was lower, with single signs containing more messages. This redundancy in information settings led to a decrease in the efficiency of travelers in obtaining information. Accurate visualization, completeness, and an appropriate amount of information are essential prerequisites for travelers to efficiently and accurately gather information.

Comparing the results obtained through EWM reveals that the evaluation of signage at Shanghai Pudong Airport was superior to that at Xi’an North Station in both the first and second experiments. This indicates that under the same evaluation criteria, the wayfinding signage at Shanghai Pudong Airport is more effective than at Xi’an North Station. Moreover, this consistency across different locations and hubs demonstrates the robustness of the proposed evaluation method. Additionally, variations in participants’ visual attention lead to changes in their overall assessments of the signage, reflecting shifts in longitudinal evaluation results. However, despite these longitudinal changes, the cross-sectional comparisons between different hubs remain consistent.

3.2.2. TOPSIS Results

The scenarios employed in the experiment represent different types of integrated transportation hubs in various cities. Even when applying the same set of methods, data collected in different experimental scenarios show variation. By comparing the ranking and weight of indicators, the varying importance of different factors within distinct transportation hubs can be observed. Regional heterogeneity may influence the importance of different evaluation indicators. Therefore, a combination of the entropy weight method and TOPSIS is utilized to rank the indicators. Relative closeness refers to the degree of proximity of the object being evaluated to the ideal target. The comparison of the importance of different indicators is expressed in terms of relative closeness (

Table 7. Relative Closeness Degree.

	Pudong		Xi’an
	Closeness	Order	Closeness	Order
Readability	0.798	3 *	0.702	3 *
Blocking rate	0.769	4	0.773	1 *
Size rationality	0.661	5	0.466	12
Brightness	0.630	8	0.556	8
Location rationality	0.632	7	0.603	7
Completeness	0.461	12	0.622	5
Conspicuity	0.659	6	0.514	9
Concise clarity	0.630	9	0.433	13
Guidance accuracy	0.546	10	0.613	6
Hierarchy	0.455	13	0.397	14
Consistency	0.510	11	0.703	2 *
Informativeness	0.937	1 *	0.480	10
Visual attention	0.830	2 *	0.639	4
Signage attention	0.287	14	0.471	11
Average attention	0.273	15	0.330	15

Note: * indicates the top three rankings.

).

By combining the results of EWM and TOPSIS, the importance of indicators was ranked. For Pudong Airport, the ranking is visual recognition > information expression > setting > visual attention. Dominant factors include the amount of information on a single sign, visual attention, and reasonable signage location. Factors of insufficient importance encompass average attention per sign, attention to single signs, and the hierarchy of signage, among others.

As for Xi’an North Railway Station, the ranking is visual recognition > setting > information expression > visual attention factors. Dominant factors include obstruction and defacement rate, coherence and consistency, and accurate legibility. Factors with insufficient importance involve average attention per sign, distinctness of signage’s primary and secondary elements, conciseness, and clarity.

It is essential to note that variations in experimental scenarios and tasks yield differences in obtained data. Through a comparison of indicator ordering and weighting, the importance of different indicators was assessed.

• Shanghai Pudong Airport: The visual attention factor received the lowest score, indicating insufficient attention during the experimental process. This is reflected in evaluations of “coherent consistency”, “clear separation of primary and secondary”, and “completeness of signage arrangement”, suggesting scattered and insufficiently coherent signage. Visual recognition and information expression factors were better evaluated, especially the amount of information on a single sign and accurate recognition. Visual recognition is crucial for travelers to obtain information efficiently, directly impacting the processing speed of guidance signage information. Accurate legibility, reflecting visual legibility, indicates reasonable signage placement and an appropriate amount of information.
• Xi’an North Railway Station: The importance of visual attention factors, especially average attention per sign, is ranked lowest. Evaluations of “simplicity and clarity”, “information quantity of a single sign”, and “priority” suggest unclear guidance signage due to excessive information, making hierarchization unclear. The large amount of information requires travelers to process more information during wayfinding, leading to lower average attention rates. The evaluation of signage visual recognition and information expression factors is better, especially regarding obstruction and defacement rate and the accuracy of recognition. Travelers prioritize eye-catching and clear signage that is accurately read and not defaced. Combining “consistency” and “visual attention”, guidance signage is set up consistently, attracting travelers’ visual attention. However, the redundancy of information on single signs hampers efficient wayfinding.

4. Discussion

Existing research has established a solid foundation for the design and optimization of signage across various application scenarios. To ensure the scientific rigor of the experimental research, a preliminary study was first conducted using methods such as questionnaires, VR surveys, and image/video analyses. This investigation demonstrated that pedestrians primarily focus on signage and gather information through visual attention during navigation. Building on this insight, visual attention metrics were integrated into the signage evaluation from an objective perspective [37]. This approach provides a clearer understanding of the extent to which pedestrians visually engage with signage features, facilitating more targeted optimization and improvement strategies. By aligning principles of directional signage design with relevant academic literature, a comprehensive evaluation framework for the visual effectiveness of signage within transportation hubs was developed.

Traditional evaluation methods, such as field survey questionnaires and on-site experiments, are considered conventional and somewhat limited in scope [38]. Cognitive experimental evaluation methods, while insightful, often exhibit a degree of subjectivity and qualitative bias [3]. Additionally, these methods generally lack a comprehensive framework for quantifying the effectiveness of various signage categories and configurations in guiding pedestrians [25]. In contrast, driving simulators, known for their high fidelity and flexibility, offer innovative perspectives for road traffic evaluation. Although VR technology could simulate real-world scenarios effectively, its use of VR headsets may induce dizziness and other adverse effects, thereby constraining the duration of experiments [39].

Research indicates that signage evaluation should integrate real-world or simulated scenarios to achieve a more accurate assessment. In transportation hubs, where the primary users of directional signage are pedestrians and travelers, it is beneficial to draw upon methodologies from driving simulations and VR experiments. Consequently, this study proposes a panoramic video-based evaluation approach. Panoramic video, with its high level of immersion, could authentically replicate real-world environments, enabling observers to make relatively objective assessments even if they are not familiar with the specific signage being evaluated [40].

Signage plays a crucial role as a reference for travelers navigating transportation hubs. Recognizing the significance of travelers’ visual attention in wayfinding behavior, the paper designed a panoramic video-based experiment to evaluate the effectiveness of wayfinding signage. The evaluation system, established from the principles of the guidance system and field investigation results, encompasses visual recognition, setup, information expression, and visual attention. Pudong Airport and Xi’an North Railway Station scenes were selected as the experimental and control groups, and multi-scene multi-task experiments were conducted. The acquired eye movement and survey data were processed using the entropy weight and TOPSIS methods.

An overall evaluation of different transportation hubs (Xi’an North Railway Station and Pudong Airport) was performed in a multi-task experiment.

In the taskless experiment, Pudong Airport and Xi’an North Railway Station received scores of 0.648 and 0.479, respectively. In the task-driven experiment, the scores were 0.228 and 0.219. Despite evaluating signage in different hubs and cities, the results demonstrated consistency and scientific validity, offering valuable insights for subsequent guidance in signage design and evaluation. These results demonstrate that the evaluation method maintains consistency and scientific validity across diverse cities and hubs. The proposed experimental scheme and evaluation methodology prove effective in assessing various types and levels of transportation hubs. The design elements of directional signage, such as color, brightness, and placement, significantly influence pedestrians’ visual attention during navigation. Consequently, evaluating pedestrian visual attention provides an indirect measure of the signage’s effectiveness and its ability to capture attention.

Regarding travelers’ visual attention, in the Pudong scene comparison, visual attention increased by 22%, while in Xi’an North Station, it increased by 25%. This suggests that travelers focus on guidance signage when seeking directions. The analysis delved into the role and differences of traveler visual attention in evaluating indicators. Key indicators influenced by visual attention include the amount of information in a single sign, accurate recognition, coherence, and blocking defacement. These variables exhibit significant changes when visual attention shifts, indicating their pivotal role in assessing signage guidance effectiveness.

The visual perception of participants serves as an indicator of the effectiveness of signage in guiding pedestrians. The results from the evaluation experiments reveal notable variations in pedestrian visual attention throughout the assessment process. Participants rated the visibility and setup of signage more favorably when their focus was on the overall environment rather than on the signage alone. However, there was greater variability in their evaluations regarding information expression, especially concerning the “amount of information per sign”. An inadequate amount of information fails to enhance signage recognition effectively, whereas excessive information may lead pedestrians to overlook critical details. These findings are consistent with existing research on tunnel signage evaluation, which supports the observed results [33]. When signs are integrated into the broader environment, pedestrians initially perceive the signage’s setup, including its position, size, and brightness. In contrast, when focusing solely on the signage, pedestrians are more concerned with directly obtaining useful information. Therefore, the amount of information presented on a sign directly impacts the efficiency of information retrieval.

In summary, the panoramic video-based evaluation method for pedestrian directional signage proposed in this study offers both significant theoretical and practical value.

• Theoretically, this study introduces an innovative approach by integrating panoramic video-based experimental design with traditional questionnaire methods. It investigates the intrinsic relationship between signage visual effectiveness and pedestrian visual attention across various dimensions, including signage appearance and configuration. By employing EWM and TOPSIS for objective evaluation, this research advances the theoretical understanding of the objective assessment of signage placement. Moreover, it establishes an experimental framework for optimizing and evaluating signage from a visual cognition perspective.
• Practically, the proposed evaluation method has been substantiated through case studies at Pudong Airport and Xi’an North Railway Station. The research and experimental validation demonstrate the correlation between the visual effectiveness of signage and pedestrian visual attention and evaluation outcomes. This provides valuable insights for traffic management, aiding in the optimization of signage placement, reduction of visual obstructions, and enhancement of signage clarity. Consequently, the method offers practical guidance for improving the efficiency of directional signage within complex transportation hubs.

5. Conclusions

Signage is essential for guiding pedestrians and travelers within transportation hubs. Given the critical role of pedestrian visual attention in navigation, this study developed an evaluation experiment for directional signage utilizing panoramic video. By merging insights from field research with established design principles for directional systems, an evaluation framework was created that addresses factors such as visibility, signage placement, information presentation, and visual attention. The experiment was conducted across multiple scenarios at Pudong Airport and Xi’an North Railway Station, which served as experimental and control groups, respectively. A multi-scenario, multi-task experimental design was implemented. Data collected from eye-tracking and questionnaires were subsequently analyzed using EWM and TOPSIS.

The results demonstrated consistency and scientific rigor in assessing signage across different cities and hubs, providing valuable insights for the future design and evaluation of directional signage. The study also examined the influence of pedestrian visual attention on evaluation metrics. From this perspective, factors such as the information volume of individual signs, readability, coherence, and issues related to obstruction and degradation were identified as key determinants influencing signage effectiveness.

Based on these insights, the following recommendations are proposed to improve directional signage:

• Optimize the Relative Position of Directional Signage: Improve the visibility of signage within the overall environment. The current lack of prominence results in significant discrepancies in obstruction rates, damage rates, and visibility assessments. Further optimization of sign size and brightness is also necessary.
• Provide Clearer and More Concise Directional Signage: Regarding the amount of information per sign, results indicate that this factor significantly affects visual efficiency. Future optimizations should focus on eliminating redundant information and reducing the amount of information on the individual. Alternatively, increasing the number of signs appropriately could help distribute information more effectively across multiple signs.

Therefore, design improvements for directional signage should be guided by pedestrian visual experience. As pedestrians navigate using directional signage, their visual focus is more concentrated on these signs. Therefore, designing and optimizing signage to appropriately engage pedestrian visual perception will enhance information retrieval and improve navigation efficiency.

The newly proposed evaluation approach based on panoramic video overcomes the limitations of realistic conditions. Considering the traveler’s visual attention factor, it offers a valuable reference for optimizing existing signage and enhancing the efficiency of traveler wayfinding from the user’s perspective. The research process focused on the restoration of the experimental scenario, subjective evaluation by the subjects, and the effectiveness assessment of guidance signage. In future work, the following enhancements to the experimental scenario are recommended for a more scientific and effective evaluation:

• Experimental Scene Selection: Limited by the available experimental time, the chosen traveler flow lines and signs lack comprehensiveness. The clarity of the panoramic video diminishes after processing. Therefore, during video acquisition, it is crucial to integrate the core functions of the hub and capture the entire traveler flow lines as thoroughly as possible. When transforming the spherical panoramic video, clarity must be ensured to accurately restore the real scene with high precision.
• Experimental Program Design: The equipment used did not fully replicate real-world conditions, leading to discrepancies between the experimental setup and reality. Moreover, the questionnaire design was limited by a small sample size, restricting the information obtained. Future experiments should enhance the use of equipment to better simulate real-world scenarios and increase sample sizes. Additionally, future studies should include scenarios from small and medium-sized cities with varying economic development levels to validate the generalizability of the evaluation method. In the design of the multi-task comparative experiment, the paper selected a large-scale comprehensive transportation hub as a comparative experimental object. In subsequent research, it is advisable to contemplate selecting scenes from small and medium-sized cities across different regions and levels of economic development. This approach will help verify and evaluate the potential universality of the experimental program.
• Data Collection and Analysis: The experimental scheme solely gathered survey evaluation data and eye movement data, employing only fuzzy calculations for the traveler visual attention index, which constrained the precision of the analysis. The information obtained from surveys is limited, and the sample size acquired is low. To enhance precision, more precise indicators should be derived by combining eye movement data and expanding the sample size. In-depth studies could incorporate the collection of electroencephalogram (EEG) and electrocardiogram (ECG) data to objectively reflect the evaluation results. Additionally, it is crucial to diversify the sample size across different demographic groups (including the elderly, the young, and special groups) to discuss and analyze potential evaluation differences.

In conclusion, this paper addresses the limitations of real technical conditions and traditional investigation methods by proposing a visual attention-based evaluation scheme using panoramic video. Through the capture of real scenes in panoramic videos and the creation of experimental scenes for replication, combined with survey and eye-tracking device data, the paper provides a more objective evaluation of the visual setting effect of guidance signage within transportation hubs. This approach offers a fresh perspective for the design and optimization of guidance signage, with significant theoretical and practical implications.