Measurement Quality Appraisal Instrument for Evaluation of Walkability Assessment Tools Based on Walking Needs

Abstract

Walking is a sustainable commute mode, and walkability is considered an essential sign of sustainable mobility. To date, many walkability assessment tools have been developed to assess the walkability conditions across the world. However, there is a paucity of comprehensive methods to assess current walkability tools based on walking needs and ensure all walking requirements are included. Thus, researchers and experts are unable to select the most comprehensive tool systematically. The present study attempts to develop a system to evaluate the quality of the existing tools. The instrument focuses on factors related to walking needs frequently observed in all types of walkability assessment tools. Hence, a pilot measurement quality appraisal instrument (MQAI) is developed and tested by a research team with planning and public health backgrounds. The final MQAI is tested by suitable reliability, criterion, and content validity tests. Most appraisal scales display moderate to high reliability for both audits and questionnaires. The MQAI appears as ready for use in several applications, including meta-analyses and systematic reviews. Additionally, the MQAI can be used by practitioners and planners to identify the most comprehensive and efficient assessment tools based on their needs.

1. Introduction

Walking is the simplest class of physical movement that benefits individual health. In addition, walking is regarded as a sustainable transport mode that benefits an individual, society, and environment [1,2,3]. Several studies focused on identifying pedestrian needs [4,5]. These studies identified a range of factors that affect pedestrian behavior and decisions. These factors can be summarized into four main groups that include accessibility [6,7,8,9,10], safety [11,12,13,14,15,16], comfort [4,5,17,18,19], and pleasurability [4,5,20]. A few studies also focused on a single dimension of walking needs. For example, Tiwari [21] explored the safety concerns of an individual while accessing metro stations, and Zakaria and Ujang [22] determined pedestrian comfort based on walking experience.

Several studies used these walking needs to develop assessment tools, including pedestrian level of service (PLOS) methods and walkability assessment tools [23,24,25,26,27,28,29,30]. Factors used in the aforementioned studies include accessibility, traffic factors, safety (from crime and traffic), geometry/environmental/footpath factors, pedestrian movement factors, aesthetics, comfort, attractiveness, functionality, destinations, environmental appearance, activity potential, shade, convenience, walking facilities, usability, and exploration.

The existing physical activity tools and walkability assessment tools aim to assess the walking environment and improve recreational spaces for health advancement in societies [31]. Walkability assessment tools use audits [13,32,33,34] and questionnaires [35,36] to collect the required data. In order to perform an audit, the streets are split into segments, and each part is examined by one or more evaluators. In audits, a set of qualitative judgments or quantitative measurements is designated for each assessment item. Like the audits, the questionnaires are effective instruments to assess pedestrian environments. The questionnaires are utilized to evaluate the perceptions of neighborhood residents towards walking and cycling facilities in their area.

According to Litman [37], walkability is considered an essential indicator of sustainable mobility. Typically, researchers and practitioners from various domains, including urban planning, transport planning, urban design, and public health have an interest in the topic of walkability. In addition, they are the main users of the walkability assessment tools. There are many walkability assessment tools, and it is a challenging task to select the best one. Furthermore, there is no guideline or a systematic manner to help these users to select the most appropriate walkability assessment tool. They need to ensure that the tool that they select to work with is comprehensive and sufficiently detailed. This is because the future investments in infrastructures may depend on this assessment. Thus, if an inappropriate tool is used, undesirable consequences will be brought about. Each type of walkability assessment tool uses certain indicators to assess the walking environment and urban design-related factors. The walking needs are extremely diverse, and thus it is important to ensure that the assessment tools consider a wide range of urban design-related factors to the maximum possible extent for assessment purposes. Consequently, there is a need to develop an instrument to appraise the strength of assessment tools to evaluate walking needs. Currently, there is a paucity of research dedicated to the measurement quality examination of walkability assessment tools [11]. The present study aims to develop a measurement quality appraisal instrument (MQAI) to evaluate walkability assessment tools based on walking needs. This paper presents the development process of the MQAI. To exhibit this process, the MQAI was applied to some walkability assessment tools and indicated the reliability, validity, and applicability of these tools. The successful development of MQAI ensures planners and researchers can efficiently employ this tool for choosing the most appropriate walkability assessment tool among the candidate tools.

2. Walking Needs

Various walking needs and their contributory urban design variables affect people’s decision to walk. Accessibility is among the most cited walking needs that must be met to motivate people to walk. Accessibility simply refers to the ability (easiness) of obtaining desired services and activities [4,6,7,8,9,10]. Several urban design factors affect the accessibility needs of walking, including, but not limited to, availability/completeness of sidewalk network, number of destinations, proximity to transit points, presence/number of barriers, and public spaces.

Safety is another important walking need that is frequently found in the literature. Safety of walking refers to whether an individual feels safe from the danger of falling due to wet conditions, the hazard of conflicts with vehicles, and the threat of crime [2,4,11,12,13,14,38]. Urban design factors that may affect safety from crime include lighting, landscape and trees, and vacant buildings. Design factors that may contribute to safety from traffic include signage, signals, and pedestrian crossings. Safety from falling also can be affected by surface, materials, and lighting.

A considerable amount of literature has been published on comfort as an important need for walking. Comfort refers to a person’s level of satisfaction, ease, and pleasure [4,5,17]. The design factors that may affect the comfort needs of walking include landscape and trees, the presence of traffic calming features, canopies, and drinking fountains. Pleasurability is also an important need for walking. Pleasurability simply refers to whether an individual experiences an enjoyable and interesting area for walking [4,5,20]. The presence of a varied streetscape, architectural elements, and outdoor dining areas can affect the pleasurability level of pedestrians.

Table 1. Walking requirements and their design considerations.

Walking Needs	Design Factors That Affect the Walking Needs	Reference
Accessibility	Presence/completeness of sidewalk network	[4,12,22]
	Presence/number of barriers	[4,39]
	Distance to destinations	[4,39,40]
	Number of destinations	[4]
	Pavement width	[24]
	Pattern of street network	[39]
	Variety and proximity of activities	[4,39]
	Connectivity between uses	[39]
	Mix land use	[12,39]
	Alternative routes	[22]
	Proximity to transit points	[22]
	Directness of walkways/routes	[22]
	Accessibility of transit stations	[22]
	Public space	[12]
Safety	Landscape and trees (buffering and safety from crime)	[26,41,42,43,44]
	Signage	[32,35,45,46]
	Bollards	[44,47,48,49]
	Lighting	[12,14,32,50,51,52]
	Signals	[32,34,53,54]
	Surface and material	[35,46,55]
	Windows facing the street	[12]
	Abandoned buildings	[12]
	Rundown buildings	[12]
	Vacant buildings	[12]
	Graffiti	[12]
	Undesirable land uses	[12]
	Driveways	[56]
	Pedestrian crossings	[57,58,59]
Comfort	Landscape and trees	[24,60]
	Sidewalk width	[49,52,61,62]
	Surface/Material	[46,52]
	Lighting	[63]
	Cleanliness	[6]
	Shading	[6]
	Presence of traffic calming features	[4,12]
	i. Roundabouts/Traffic circle
	ii. Medians
	iii. curb bulb-outs
	Length of blocks	[12,39]
	Buffer	[12,39]
	Arcades	[4]
	Canopies	[4]
	Number of lanes	[12]
	Street width	[12]
	Pavement treatment	[12]
	Drinking fountains	[64]
	Bench and sitting areas	[35,65]
	Slope	[52,66]
Pleasurability	Landscape and trees	[12,67]
	Presence of a varied streetscape	[4]
	Mixed uses	[4]
	Architectural elements	[4]
	Historic or unique architecture	[4]
	Color	[4]
	Presence of public space	[4]
	Outdoor dining areas	[4]

illustrates the walking needs and the urban design factors that affect these needs.

The existing walkability assessment tools have various factor classifications. In walkability assessment tools, the major groups of assessment items are street facilities, sidewalk characteristics, land use, and road attributes. Street facilities include signage, signals, drinking fountains, surveillance, and items related to the disabled [33,34,68]. Sidewalk characteristics include items such as sidewalk completeness, the width of the sidewalk, presence/number of barriers (obstacles), and surface/material of the sidewalk [69,70,71]. Land use is another frequently used grouping that contains a mixture of land use, undesirable land uses, and destinations [72,73]. The walkability assessment tools also use items related to road attributes, including traffic calming features, street width, cleanliness, lighting, and directness of walkways/routes [71,74].

Table 2. Intersections between the walking needs and the major assessment categories.

	Sidewalk		Land Uses and Destinations			Street Facilities													Road Attributes
	Sidewalk Network	Path Condition	Mixture of Land Use	Undesirable Land Uses	Destinations	Signage	Signals	Drinking Fountains	Bollards	Landscape/Trees	Buffers	Benches/Sitting Areas	Surveillance	Items Related to	Disabled	Streetscape Characters	Driveways	Transit Points	Traffic Calming	Features	Road Attributes	Qualitative	Characteristics	Network Design
Accessibility	1.2		2.26		1.7									3				32						26.32
	32				26
Safety				2		5.10	5.15		3.22	8.14	21.22		2	3			38					11		34.35
						16.28	27.30		25.31	22.24														37
Comfort	4.17	4.28						39		20.29	2.26	16.33		3					1.2		2	19.12		2.26
	18.25	36
Pleasurability			1							2.23				3		1						1

1 = Alfonzo [4]; 2 = Alfonzo, Boarnet, Day, McMillan and Anderson [12]; 3 = Asadi-Shekari, Moeinaddini and Zaly Shah [48]; 4 = Azemati, Bagheri, Hosseini and Maleki [52]; 5 = Clifton, Livi Smith and Rodriguez [32]; 6 = Crews and Zavotka [50]; 7 = Cubukcu [40]; 8 = Cui, Allan, Taylor and Lin [42]; 9 = Foster and Giles-Corti [51]; 10 = Funk [45]; 11 = Haans and de Kort [14]; 12 = Handy and Clifton [6]; 13 = Harkey and Zegeer [55]; 14 = Hernandez [41]; 15 = Karim and Azmi [53]; 16 = Kihl, Brennan, Gabhawala, List and Mittal [35]; 17 = Kim, Choi and Kim [62]; 18 = Krambeck [61]; 19 = Landis, Vattikuti, Ottenberg, McLeod and Guttenplan [63]; 20 = Lee, Jang, Wang and Namgung [60]; 21 = MacNeil [43]; 22 = Matan and Newman [44]; 23 = Rahimiashtiani and Ujang [67]; 24 = Samarasekara, Fukahori and Kubota [26]; 25 = Samarasekara, Fukahori and Kubota [49]; 26 = Sapawi and Said [39]; 27 = Slater, Nicholson, Chriqui, Barker, Chaloupka and Johnston [54]; 28 = Southworth [46]; 29 = Talavera-Garcia and Soria-Lara [24]; 30 = Troped, Cromley, Fragala, Melly, Hasbrouck, Gortmaker and Brownson [34]; 31 = Van Cauwenberg, Van Holle, Simons, Deridder, Clarys, Goubert, Nasar, Salmon, De Bourdeaudhuij and Deforche [47]; 32 = Zakaria and Ujang [22]; 33 = Galanis and Eliou [65]; 34 = Keat, Yaacob and Hashim [57]; 35 = Kolbe-Alexander, Pacheco, Tomaz, Karpul and Lambert [59]; 36 = Monteiro and Campos [66]; 37 = O’Connor, Borscheid and Reid [58]; 38 = Otak [56]; 39 = Sarkar [64].

presents walking needs-related factors based on the major factor classifications in the existing walkability assessment tools. The walking needs information obtained from the literature and summarized in Table 1 and Table 2 were used to develop a comprehensive instrument to assess current tools based on walking needs. This instrument can assess the quality of the existing walkability assessment tools and determine their capability for assessing pedestrian environments. Such an instrument also can act as a decision-making system for selecting the most appropriate assessment tool for evaluating the walking environments.

3. Methods

As previously mentioned, this paper shows the development process of the MQAI. This process included two main parts: (1) pilot version development and (2) final version development. Each part involved a series of assessments and techniques. The development process of MQAI is indicated in Figure 1.

3.1. Identifying Walking Needs and Developing the Pilot MQAI

A literature review has been conducted to identify the walking needs and their widest range of contributory urban design factors. The walking needs information extracted from this literature (refer to Table 1 and Table 2) were used to develop a pilot MQAI (refer to Appendix A) to assess the current tools based on walking needs.

Table 3. Characteristics of MQAI.

Type of Appraisal	Scale	Question No.
Sidewalk	Sidewalk network and path condition	1–2
Land uses and destinations	A mixture of land uses, undesirable land uses, and destinations	3–5
Street facilities	Signage, signals, drinking fountains, landscape and trees, bollards, buffers, benches and sitting areas, surveillance, items related to disability, streetscape characters, driveways, and transit points	6–17
Road attributes	Traffic calming features, road attributes, network design, and qualitative characteristics	18–21

lists the key characteristics of the MQAI. This tool is based on a pointing system in which each point corresponds to a specific condition. In this system, the worst and best conditions receive the lowest and greatest points, respectively. This method facilitates a systematic comparison among the walkability assessment tools and allows for determining the tools’ capability for evaluating the walkability. To assess each item, the evaluator must select ‘no assessment’ (determines that the tool does not assess the indicator); ‘simple assessment’ (determines that the tool simply assesses the availability of an indicator and does not assess the quality of indicator); ‘partial assessment’ (determines that the tool assesses the availability in addition to the quality but does not provide a complete assessment for the quality); and ‘complete assessment’ (determines that the tool presents a complete assessment (availability and quality) for the indicator). The ’no assessment’, ’simple assessment’, ’partial assessment’, and ’complete assessment’ conditions receive points of zero, one, two, and three, respectively. These four levels of responses allow for simultaneously assessing both availabilities of design factors and their assessment quality in the tools. The score of each measurement scale is computed by the sum of the marks assigned to the different items. Appendix A shows the scoring pattern and related explanations.

To investigate the content validity of the proposed MQAI, some meetings were held with a panel of experts which included two experts in urban transport planning and public health. The outcomes of these meetings were minimal changes to the content of some scales and/or the explanation attached. The pilot version of MQAI was made through the results of this step.

A criterion validity test was conducted in this step. Two pedestrian environment assessment tools, including one audit and one questionnaire, were assessed utilizing the pilot version of MQAI by the research team (authors). Each member of the research team was benchmarked relative to the team leader (first author). The average level of agreement was 41.5%.

Once the assessment of criterion validity of the MQAI pilot version was completed, the outcomes of this evaluation were discussed in a series of meetings in which both the research team and experts were involved. These meetings engaged the experts in discussion and the developing of a refined list of suitable MQAI appraisal items. During the meetings, the research team and experts confirmed the purpose and scope of the MQAI. They also ensured that the widest range of appraisal items was included in the proposed instrument. Thus, a few changes were implemented, such as adding more explanations to the description of the responses to clarify the differences between answer categories in a better way (refer to

Table 4. An example of an added explanation to a given question in MQAI.

Answer	Description	Point
No assessment	The tool does not assess the condition of the path	0
Simple assessment	The tool simply assesses the path condition * in the study area	1
Partial assessment	The tool assesses the path condition, and one of the following path condition issues: (1) material ** used; and (2) slope ***	2
Complete assessment	The tool assesses the path condition and all the following path condition issues: (1) material used; and (2) slope	3

* Poor (several weeds, breaks, and holes), moderate (a few weeds, breaks, and holes), good (very few weeds, breaks, and holes), under repair. ** Flat segmented concrete slabs, paving stones, Portuguese mosaic, rustic natural stones, slippery material (smooth ceramic tiles), rough material (hydraulic tiles, interlocked blocks, flattened concrete), regular, firm, antiskid, and ant vibration material (high strength paving). *** Flat or gentle, moderate slope, steep slope.

). Additional improvements included adding an instruction to respond to the appraisal items. Step-by-step instructions were provided to aid users in selecting a suitable answer concerning ‘No, simple, partial, and complete’ (refer to Appendix A). Additionally, a graphical scale was provided to help the users recognize the right response (refer to Figure 2).

3.2. Final Version Development

The research team and panel of experts assessed the significance of each tool item. They rated the importance of the items by utilizing a five-point scale varying between ’not important’ and ’very important’. The median score for each item was calculated to determine the weight of the items. In order to gain the consensus of the research team, the team computed the agreement level for the importance of every factor. Then, the weight for each item was adjusted based on the number of items in each category.

The formula that was utilized is [weight − expected weight]. The expected weight is the score that is assigned if the items equally contributed to a category. For instance, if it is required to weigh two items, the expected weight is 2.50 for each; and if it is needed to measure four items, then the expected weight is 1.25. The inter-quartile range (IQR) is calculated for these modified weights to assess the degree of consensus among the evaluators on the scored importance of items. Items with an IQR < 1 correspond to a high level of consensus among the evaluators.

The final version of MQAI was tested for criterion validity and reliability. The reference degree of correlation and agreement for individuals with a background and familiarity with urban planning and urban design was investigated to assess the criterion validity of the MQAI. For each rater, the agreement level was calculated with respect to the leader of the research team. A total of eight students who registered for a Master of Science (advanced urban planning course) participated in this step. Two tools were selected by the team leader and were classified based on the MQAI% interpretation section (Appendix A) as poor (20 ≤ MQAI% < 40) and regular (40 ≤ MQAI% < 60). A tool was given to each student, and they were asked to complete the assignment in four days.

In order to test the reliability, two raters were asked to evaluate six walkability assessment tools (three audits and three questionnaires). The users of walking assessment tools are mainly from the domains of urban planning, transportation planning, and public health. Thus, two raters were selected, namely an urban and transport planner and a public health expert. The main goals of this step were: (1) to verify the inter-rater degree of agreement for each of the four levels of answers employed in the MQAI; and (2) to assess the inter-rater degree of agreement for each of the six tools. The inter-rater reliability was tested by using Kappa, which is a statistical measure of inter-rater reliability.

4. Results

Based on the IQR definition, an IQR of less than one indicates a high level of agreement, and an IQR of more than one indicates a low level of agreement. Thus, sixteen factors exhibited high levels while five factors exhibited moderate levels of consensus (

Table 5. Relative weightings and level of consensus on the items.

	Median	Final Weight = [Weight − Expected Weight *]	Relative Weight (Normalized Weight)	IQR
Sidewalk
Sidewalk network	5	2.50	0.50	0
Path condition	5	2.50	0.50	0.5
Land use and destinations
Mixture of land use	3	1.33	0.19	0.5
Undesirable land uses	4	2.33	0.33	0.5
Destinations	5	3.33	0.48	0
Street Facilities
Signage	3	2.58	0.07	1
Signals	4	3.58	0.09	1
Drinking fountains	3	2.58	0.07	0.5
Bollards	4	3.58	0.09	0
Landscape and trees	4	3.58	0.09	1
Buffers	4	3.58	0.09	0.5
Benches and sitting areas	4	3.58	0.09	0.5
Surveillance	4	3.58	0.09	0.5
Items related to disabled	4	3.58	0.09	0.5
Streetscape characters	4	3.58	0.09	1
Driveways	3	2.58	0.07	0.5
Transit points	3	2.58	0.07	1
Road Attributes
Calming features	4	2.75	0.23	0.5
Road attributes	4	2.75	0.23	0
Qualitative characteristics	4	2.75	0.23	0
Network design	5	3.75	0.31	0.5

High level of agreement (IQR < 1); moderate level of agreement (IQR = 1); * low level of agreement (IQR > 1). * Expected weight: sidewalk factors = 2.5; land use and destination factors = 1.67; street facilities factors = 0.42; road attribute factors = 1.25.

). All items, including ’sidewalk’, ’land use and destinations’, and ’road attributes’, exhibited high levels of agreement while those items with moderate levels belonged to the ’street facilities’ category.

The final version of MQAI was tested for criterion validity and reliability. As shown in

Table 6. Baseline degree of agreement with respect to the team leader (criterion validity testing).

	Questions	Testing of MQAI Final Version
		Poor Tool	Regular Tool	Overall
Sidewalk	1–2	75%	75%	75%
Land use and destinations	3–5	83%	75%	79%
Street facilities	6–17	92%	73%	83%
Road attributes	18–21	94%	81%	88%
Overall	1–21	89%	75%	82%
Spearman correlation	1–21	0.92 *	0.78 *	0.85

* Correlation is significant at the 0.01 level (2-tailed).

, the total baseline of agreement level between the evaluators and the team head was 82%. The lowest agreement belonged to the sidewalk scale (75%). Agreement values for the other three scales were 79% for land use and destinations, 83% for street facilities, and 88% for road attributes. The Spearman correlations were 0.78 for the regular tool and 0.92 for the poor tool. The average MQAI% for the tools was 38% for the poor tool and 43% for the regular tool. The difference in MQAI% between the ’poor’ and ’regular’ tools were statistically non-significant at the 5% level. Additionally, there was no statistically significant difference in MQAI% between the tools assessed by the research team leader and the tools assessed by the individuals (p-value = 0.3 for the poor tool; p-value = 0.1 for the regular tool).

Table 7. Inter-rater degree of agreement for each of the tool types and levels of answers.

Based on the Assessments of Three Audits
	No Assessment	Simple Assessment	Partial Assessment	Complete Assessment
No assessment	80%	13.3%	6.7%	0.0%%
Simple assessment	19.2%	65.4%	15.4%	0.0%%
Partial assessment	0.0%	21.4%	71.4%	7.1%
Complete assessment	0.0%	0.0%	37.5%	62.5%
Based on the Assessments of Three Questionnaires
	No Assessment	Simple Assessment	Partial Assessment	Complete Assessment
No assessment	100%	0.0%	0.0%	0.0%
Simple assessment	21.7%	69.6%	8.7%	0.0%
Partial assessment	0.0%	70%	30%	0.0%
Complete assessment	0.0%	0.0%	60%	40%

reveals the inter-rater agreement level for every of the four levels of response employed in the MQAI.

Table 8. Inter-rater degree of the agreement for each of the six tools.

		Audit 1		Audit 2		Audit 3
		High a	Low b	High	Low	High	Low
Sidewalk	1–2	2	0	1	1	1	1
Land use and destinations	3–5	2	1	1	2	1	2
Street facilities	6–17	11	1	7	5	8	4
Road attributes	18–21	4	0	3	1	3	1
		Questionnaire 1		Questionnaire 2		Questionnaire 3
		High	Low	High	Low	High	Low
Sidewalk	1–2	2	0	2	0	2	0
Land use and destinations	3–5	3	0	1	2	1	2
Street facilities	6–17	9	3	7	5	10	2
Road attributes	18–21	3	1	3	1	3	1

^a Number of items with percent agreement ≥ 0.75. ^b Number of items with percent agreement < 0.75.

presents the reliability data by appraisal type and includes the number of questions evaluated within each component. With respect to the questions assessed in audits and questionnaires, averages of 69.84% and 73%, respectively, corresponded to a high agreement (≥75%) between the raters. The aggregated results of the inter-rater agreement level for each of the tool types are shown in

Table 9. Aggregated results for the inter-rater degree of agreement for the assessed tools.

		Audits		Questionnaires
		Weighted Kappa	95% CI	Weighted Kappa	95% CI
Sidewalk	1–2	0.57	0.02 to 1.00	1.00	1.00 to 1.00
Land use and destinations	3–5	0.45	0.13 to 0.77	0.59	0.21 to 0.96
Street facilities	6–17	0.59	0.37 to 0.80	0.62	0.47 to 0.76
Road attributes	18–21	0.79	0.50 to 1.00	0.79	0.60 to 0.97

Strength of agreement = K < 0.20: Poor; 0.21 < K ≤ 0.40: Fair; 0.41 < K ≤ 0.60: Moderate; 0.61 < K ≤ 0.80: Good; 0.81 < K ≤ 1.00: Very good.

. The weighted Kappa values for the four scales varied based on the tool type, and the K values for the audits were in the moderate to good range. Concerning the questionnaires, the K values ranged from fair/moderate to very good. The overall inter-rater reliability for the audits and questionnaires were 70% and 73%, respectively.

5. Discussion

The baseline agreement level for the overall instrument was 82% for persons with a background in urban planning and urban design with respect to the team leader. The land use and destinations, street facilities, and road attributes scored the agreement levels in the range of 79–88%. The sidewalk scale had the lowest value, that is, a 75% agreement level. The main reason for this is that this scale includes only two items; therefore, missing an item will have a larger influence on the agreement level.

The improvement of the final version of MQAI compared to the pilot version was demonstrated through testing the final version with two raters with planning and public health backgrounds. This improvement might be related to adding instructions and the items’ details. A simple check on the reliability results shows that the Kappa value is different for the same scale in audits and questionnaires. For example, the sidewalk attained a lower K value in audits than questionnaires. A possible explanation for this is the inherent difference of assessment in audits and questionnaires besides the dearth of knowledge in a specific field of proficiency. During testing of the MQAI instrument, the team noted that raters had difficulty choosing the ’partial’ response. However, the raters did not experience any difficulty in assigning other response categories. The interpretation skill of raters was further significantly improved through in-depth training and supervision.

The results also showed that ’poor’ tools are easier to assess than regular tools. The total scoring for a ’poor’ tool by raters was very similar to that of the team leader. Based on the classification of the tools proposed in this study, the ’poor’ tool represents a tool that considers a few numbers of urban design factors. Hence, the raters were required to easily score items as ’no’ or ’simple’.

Both researchers in practice and academia can employ the MQAI to select the most suitable walkability assessment tool. The walkability assessment tools help decision-makers to identify shortcomings in the living environments. Decision-makers then conclude about the improvement strategies for a living environment with undesirable walking conditions. These strategies may include financial and cultural aspects, which may impact the everyday life of the residents. It is vital that a sufficient amount of investments be allocated to an area with inadequate walking conditions. A better walking infrastructure encourages people to walk and, in turn, increases the overall walking level of residents in a neighborhood. Thus, choosing a suitable walkability assessment tool that assesses the walking environment accurately is of great interest. Moreover, this can impact the plans for improving the walking conditions in a neighborhood indirectly. The employment of MQAI enables practitioners to (1) classify the walkability assessment tools, (2) select the most suitable one, and finally (3) identify walkability shortcomings within neighborhoods using the selected tool.

Researchers in academia also can benefit from the MQAI. Researchers in the domains of urban planning, transport planning, and public health need a comprehensive tool for assessing the walkability condition in a certain area and link this condition with the overall walking level in that area. Typically, this relationship is assessed using traditional statistical methods. However, the abundance of walkability assessment tools, in both the forms of audits and questionnaires, makes it challenging for these researchers to pick the most appropriate one, which can truly reflect the walking condition within a certain area. Thus, the MQAI can help them choose the most comprehensive tool that can capture the details of the walking environment and find the associations of this environment and overall walking and physical activity levels.

6. Conclusions

In recent decades, walkability assessment tools have been developed to assess the suitability of a walking environment for pedestrians. These tools used numerous environmental factors in order to assess the built environments. To date, several reviews were published on walkability assessment tools, and they highlighted challenges faced by extant studies [11,69,70,71]. However, there is a paucity of a system for assessing the walkability assessment tools based on walking needs. The present study developed and tested an instrument to appraise walkability assessment tools based on walking needs. The main goal of the proposed instrument is to assess whether the walkability assessment tools consider the walking needs and urban design-related factors. This tool can serve as a decision-making system for researchers and practitioners to select the most appropriate assessment tool for evaluating the walking environment.

The present instrument can be used for meta-analyses and systematic reviews. This instrument is easy to use for planners and public health experts. The MQAI can aid practitioners and researchers in selecting the tool to assess the pedestrian environments in both the neighborhood and street scale based on their priorities. The instrument considers the majority of the walking needs to assess the existing tools. However, the planners can select the required items based on their priorities and adjust the proposed MQAI based on their selected items. Additionally, the instrument can serve as a base to develop future walkability assessment tools. The MQAI can be utilized to decide whether the design of a new walkability assessment tool adheres to the walking needs of diverse pedestrian groups. The MQAI did not perform the reliability and validity tests on virtual assessment tools. However, to keep abreast with new technological advancements, this tool also can be employed to assess the virtual assessment tools, which were recently released. Additionally, the methodology employed in this study can be followed to develop similar tools for assessing the virtual walkability/bikeability tools. The MQAI can also inspire future decision-making tools to select the best assessment tools that involve physical environment indicators.