*2.4. Step 3: Validation of the Objectives, Criteria, and Metrics of the RAF*

In this step, the RAF was submitted to stakeholders to identify knowledge gaps and improvement opportunities. The RAF validation includes the involvement and participation of stakeholders with different contexts, in a different resilience development level and with diverse NBS in the city, as water utilities or municipalities. The proposed dimensions, resilience objectives, criteria, and metrics were analysed with the stakeholders during working sessions. Metrics determination for participating cites and case studies is also carried in this step, taking into consideration the available information. In addition, a survey was conducted to determine the RAF metrics' relevance and the feasibility of application to their own cities.

The validation step includes (i) RAF submission to stakeholders to identify improvement opportunities and gaps, (ii) RAF application to participating cities, (iii) determination of metrics relevance and feasibility of application to cities, and (iv) complete RAF application to case studies.

Seven participating cities have contributed to the RAF validation steps—namely, Almada (Portugal), Coimbra (Portugal), Lisbon (Portugal), Porto (Portugal), Barcelona (Spain), Bristol (the United Kingdom), and Vancouver (Canada). The participating cities present different challenges regarding urban resilience and NBS. The participation of cities allows to validate the RAF taking into consideration different international and urban context, city dimension and management, NBS management (e.g., private, public), social involvement, and awareness, among other factors. Stakeholders from water utilities, municipal council and private organisation participated in the working sessions. A total of eight organizations validated the RAF.

#### *2.5. Step 4: Definition of Reference Values for Each Metric*

Reference values for each metric were defined to assess the NBS contribution to urban resilience on a normalized scale. The definition of reference values was based on existing literature review, regulations, standards and available assessment frameworks. An overall assessment will be proposed to measure the NBS contribution to urban resilience.

Reference values allow metric´s classifications by means of a judgment (e.g., satisfactory or unsatisfactory performance). Comparing the results of a metric with its reference values allow to pinpoint the existing problems and to monitor the implemented solutions.
