Exploring the Severity Strata of Disease Activity and Repigmentation in Vitiligo Based on Validated Physician Global Assessment (PGA) Scores

Abstract

Background/Objectives: There is currently no guidance on how to interpret the global degrees of activity (worsening) and repigmentation (improvement) in vitiligo. Stratification into global degrees can be completed for static evaluations (e.g., visible disease activity signs) and dynamic assessments (e.g., evolution over time). For the latter, the Vitiligo Disease Activity Score (VDAS_15&60) and Vitiligo Disease Improvement Score (VDIS_15&60) were recently validated. Methods: In the current study, a Physician Global Assessment (PGA) for disease activity (worsening) and repigmentation (improvement) was evaluated for validity (construct) and reliability (inter- and intrarater) based on a photo set of 66 patients. Subsequently, the PGA activity (worsening) and repigmentation (improvement) were used to stratify the Vitiligo Extent Score plus (VESplus), VDAS_15&60 or VDIS_15&60 into three global categories (slightly, moderately and much worse/improved), based on ROC analysis. Results: For the VESplus, cut-off values for the categories ‘slightly, moderately and much worse’ were >0.3%, >27.71% and >128.75% BSA (relative changes in the affected total BSA), respectively. For the categories ‘slightly, moderately and much improved’, they were >0%, >4.87% and >36.88% BSA (relative changes in the affected total BSA), respectively. The optimal cut-off values of the number of active (VDAS₁₅) body areas were >0 areas for slightly worse, >2 areas for moderately worse and >7 for much worse. For VDIS₁₅, the cut-off values for slightly improved and moderately improved were >0 and >1. For VDAS₆₀ and VDIS₆₀, the cut-off points were >0.5, >3, >9.5 and >0.5 and >1.5, respectively. The results should be interpreted with caution in patients with extensive vitiligo due to the rather limited disease extent of the included patient population (VESplus (median: 3.2%)). Conclusions: This research will aid in the development of more detailed international definitions.

1. Background

One of the most important outcomes in the decision-making process for vitiligo treatments is disease activity [1]. It can be assessed by the evaluation of clinical signs of disease activity at a single point (‘static’ assessment) or by the evaluation of changes over time (‘dynamic’ assessment) based on two time points. For dynamic assessments of disease activity, we introduced recently the Vitiligo Disease Activity Score (VDAS_15&60) and Vitiligo Disease Improvement Score (VDIS_15&60) [2]. It is based on the assessment of changes over time within 15 body regions. The VDAS₆₀ and VDIS₆₀ (score 0–60) are the more detailed (extensive) versions of the score in which also the degree of worsening or improvement on a scale from +4 to −4 for each body area has been added in addition to the number of locations that are improving or worsening (VDAS₁₅ and VDIS₁₅). There is still a need to define and interpret the outcomes of these instruments (e.g., “what numeric outcome represents a moderately or very active vitiligo?”). This interpretation is important as the management of vitiligo treatment is heavily reliant on it. Moreover, in the context of clinical trials, we need references for the interpretation of the results. Converting outcomes into categories (stratification) can also be useful for effectively comparing the results of different studies, especially when different instruments are used for the same outcome domain. Furthermore, categories can be used to define inclusion criteria for clinical trials and clinical profiling in epidemiological studies.

The categorization of an instrument’s numerical results is typically based on an ‘anchor question’ (e.g., Physician Global Assessment) that preferably has been validated in advance [3]. While the PGA for disease extent has been validated, this is not the case for PGAs assessing activity (worsening) and repigmentation (improvement) [4].

The first objective of this study was to evaluate the construct validity and inter-/intrarater reliability of a Physician Global Assessment (PGA) score for disease activity (worsening) and repigmentation (improvement). The second objective was to define the activity strata (“slightly/moderately/much worse/improved”) for the VESplus, VDAS_15&60 and VDIS_15&60 based on the validated PGAs (for improvement and worsening).

2. Materials and Methods

2.1. Study Design and Ethics

The research was carried out at Ghent University Hospital’s dermatology department. The ethics committee approved the study (reference number Ghent: B670201421409), and patients participating in the study provided written informed consent. Our study was designed and reported using the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist as a guide [5,6]. Based on a series of clinical images, raters were asked to complete independendently the VDAS_15&60 and VDIS_15&60, as well as a PGA (5-point scale for worsening and improvement). The instruments were used on paper for the scoring sessions [2]. The raters were instructed to score even the slightest change in extent that could be identified (PGA score of 1). The VESplus scores were scored by 1 rater at baseline and follow-up.

2.2. Raters and Participants

All raters in this study were clinicians from the dermatology department at Ghent University Hospital, with varying levels of experience. The group included five dermatology residents, one dermatologist and one vitiligo expert. In the preceding pilot sessions with the same 66 patients, three of seven raters were involved in scoring the PGA (worsening and improvement), VDAS_15&60 and VDIS_15&60. However, an interval of many months (>3) was included between the scoring rounds of the pilot sessions and the eventual definitive (real) scoring rounds to avoid any recall bias.

Patients who visited our clinic and provided written informed consent for the use of their photographs between September 2018 and September 2019 were chosen. Patients of all ages, with or without treatment, and a clinical diagnosis of non-segmental vitiligo, were included if they had at least two clinical photo sets taken 6 (±2) or 12 (±2) months apart. Patients with segmental vitiligo were not eligible.

2.3. Selection of Photographs

Two sets of clinical photos were chosen with a 4–14 month interval for each patient. In those who had multiple photo sets that met this criterion, one of the two intervals (6 (±2) or 12 (±2) months) was randomly assigned to ensure that each interval had an equal number of patients. The selected photo sets were arranged on Microsoft PowerPoint (ver.2019, 2020) slides, with each slide featuring two photographs of a single area of the body affected by vitiligo, captured on two different dates. Only body parts that were present in both sets of photos were used. Photographs were taken in a standardized manner during clinical practice, including both UV and non-UV images.

2.4. Scoring Sessions

The order of the slideshows was randomly determined using Research Randomizer (https://www.randomizer.org) for repeating scoring rounds. Prior to the first scoring round, all assessors received a training session of at least 15 min, and written instructions were also provided.

2.5. Validity and Reliability of PGA for Disease Activity (Worsening) and Repigmentation (Improvement)

Construct validity of the PGA activity (worsening) and repigmentation (improvement) were evaluated by testing at least 4 predefined hypotheses (Table S1). The VDAS_15, VDIS₁₅ and VESplus (all validated instruments) were used as comparator instruments for construct validity. Previous studies have demonstrated good construct validity and reliability for the VDAS₁₅ and VDIS₁₅ and good construct validity, reliability and responsiveness for the VESplus [2,7,8]. Two investigators (NvG and RS), who were involved in the preceding VDAS_15&60 and VDIS_15&60 validation studies [2], formulated a draft of the hypotheses. To prevent any potential bias, an external investigator (AW), who was not involved in the data collection for the PGA activity (worsening) score and repigmentation (improvement), subsequently evaluated the draft of the hypotheses. The magnitude of the relationships was then determined. This investigator consulted a second external investigator (AL) for the final approval. Construct validity was considered sufficient if at least 75% of the hypotheses agreed with the results. To assess the reliability of the PGA activity (worsening) and repigmentation (improvement), inter- and intrarater reliability were evaluated. Raters repeated the scoring rounds with an interval of at least 2 weeks, based on the same pictures, ensuring a stable clinical situation.

2.6. User Friendliness

After completion of the scoring rounds, all raters evaluated the user-friendliness of the PGA scores on a 0–10 scale.

2.7. Stratification of VESplus, VDAS_15&60 and VDIS_15&60

The validated PGA for activity (worsening) and repigmentation (improvement) was subsequently used as an anchor to stratify the VESplus, VDAS_15&60 or VDIS_15&60, respectively. By this approach, the VESplus, VDAS_15&60 and VDIS_15&60 were compared (anchored) to the results of PGA activity (worsening) and repigmentation (improvement) collected at the same time. Subsequently, ROC analyses were used to define the thresholds for categories (strata).

2.8. Statistics/Data Analysis

Statistical analyses were conducted using IBM SPSS Statistics version 26 and Medcalc 19.8. To determine interrater reliability of the PGA activity (worsening) and repigmentation (improvement) score, a two-way random, absolute agreement, single-measures intraclass correlation coefficient (ICC) was used. For intrarater reliability, a two-way mixed, absolute agreement, single-measures ICC was used. To assess interrater and intrarater reliability, an ICC of 0.75 or higher was considered excellent, 0.6 to 0.74 was good, 0.4 to 0.6 was fair and less than 0.4 was poor. Construct validity was tested using Spearman’s rho correlation. Global strata (cut-off points) for the VESplus VDAS_15&60 and VDIS_15&60 were based on Youden’s index assessed by ROC analyses using MedCalc 19.8 software (Medcalc, Mariakerke, Belgium). Missing values were excluded from the final analysis. In all cases, the significance level was set at p < 0.05.

3. Results

3.1. Raters and Participants

The first PGA activity (worsening) scoring round was completed by seven raters and the retest by five raters. A total of 66 patients were included for the scoring rounds (Fitzpatrick skin type II: 14 (21.2%), III: 39 (59.1%), IV: 2 (3.0%), V: 1 (1.5%) and VI: 1 (1.5%), unknown: 9 (13.6%); female/male: 57.6%/42.4%). The interval between both photo sets was 6 (±2) months in 34 patients and 12 (±2) months in 32 patients. The total body surface area affected (measured by the VESplus) varied between the patients (range: 0.01–36.43%, median (mean) VESplus: 1.3% (3.2%)). The mean age at inclusion was 37 (median: 39). The mean number of involved areas at baseline was 5.89 (median was 6).

3.2. Validity PGA Activity (Worsening) and Repigmentation (Improvement) Scores

Construct Validity

Table S1 contains the hypotheses and their corresponding results. Sufficient evidence for construct validity was provided for PGA activity (worsening). All four hypotheses were confirmed for PGA activity (worsening) and 3/4 hypotheses for repigmentation (improvement). As expected, the correlation between the change in affected body surface area (VESplus%) and PGA activity (worsening) and PGA repigmentation (improvement) was high (rho = 0.832 (95%CI: 0.736–0.896), and rho = 0.554 (95% CI: 0.353–0.705), both p < 0.001) (Table S1 and Figure 1). There were also very strong correlations between the PGA activity (worsening)/repigmentation (improvement) and the VDAS₁₅ or VDIS₁₅, respectively (rho = 0.956 (95%CI: 0.928–0.973) and rho = 0.839 (95% CI: 0.746–0.900), both p < 0.001) (Table S1 and Figure 2).

A multiple regression model revealed that the difference in disease extent (ΔVESplus) between the two time points as well as the number of changing locations were both independently contributing to all PGA values (worsening p = 0.006 and p < 0.001 and improvement p = 0.004 and p = 0.004, respectively). This combination of two factors (change in disease extent (ΔVESplus) and number of changed locations) explains 77.1 percent and 37.2 percent (adjusted r2) of the PGA activity (worsening) and repigmentation (improvement), respectively.

3.3. Reliability of the PGA Activity (Worsening) and PGA Repigmentation (Improvement) Score

3.3.1. Interrater Reliability

The ICC for interrater reliability of the PGA activity (worsening) was 0.707 (95% CI: 0.620–0.788). For PGA repigmentation (improvement), it was 0.701 (95% CI: 0.615–0.784).

3.3.2. Intrarater Reliability

The ICC for intrarater reliability PGA activity (worsening) was 0.770 (95% CI: 0.721–0.811), and for PGA repigmentation (improvement), the ICC was 0.726 (95% CI: 0.670–0.773). The Bland–Altman plots showed no signs of proportional or systematic error (Figure 3).

3.4. User-Friendliness

The PGA activity (worsening) and repigmentation (improvement) were both scored as user-friendly (for both, mean: 7.83; median: 8 (range: 7–9)).

3.5. Stratification of the Percentage Change in VESplus According to the PGA Activity (Worsening) and Repigmentation (Improvement)

Small changes (relative changes in total affected BSA) in worsening (>0.3%) were sufficient for a PGA activity (worsening) score of 1 (=slightly worsened) (Figure 4). Worsening of the vitiligo area by more than 27.71% and 128.75% was considered as moderate (PGA = 3) and much (PGA= 4–5) worsening, respectively. These values were lower for repigmentation, as an improvement of more than 0% resulted in a designation of slightly improved and more than 4.87% (total body) resulted in moderately improved. Repigmentation of more than 36.88% resulted in a PGA score of much improved. Nonetheless, these results should not be generalized to patient populations with different vitiligo extent. In a multiple regression model, both the relative and absolute improvement/worsening were independent predictors of the PGA improvement/worsening (p = 0.004; p = 0.004 and p = 0.004; p = 0.035, respectively). This indicates that the cut-off values for mild/moderate/much improvement will be higher and for ‘mild/moderate/much’ worsening will be lower in patients with more extensive disease.

3.6. Stratification of VDAS (VDAS₁₅ and VDAS₆₀) and VDIS (VDIS₁₅ and VDIS₆₀)

Figure 4 shows the optimal cut-off points and confidence interval of the VDAS_15&60 and VDIS_15&60 per three categories based on the respective anchor question (PGA activity (worsening) and repigmentation (improvement)). The optimal cut-off values of the total number of active body areas (=VDAS₁₅) between the three global categories (slightly worse/moderately worse/much worse) were >0, >2 and >7 and >0.5, >3 and >9.5 for the outcomes generated by the VDAS₆₀. For VDIS₁₅, slight and moderate improvement (repigmentation) corresponded to >0 and >1, and for VDIS_60, >0.5 and >1.5, respectively. Due to the small number of patients with much improvement in the dataset, the cut-off value for much improvement could not be calculated reliably.

4. Discussion

This study successfully validated the PGA score for vitiligo activity (worsening) and repigmentation (improvement). Evidence was provided for the reliability and construct validity. Additionally, the user-friendliness for the PGA activity (worsening) and repigmentation (improvement) score was favorable. A PGA is a commonly used instrument in clinical trials and may be used to define the severity strata of an instrument’s generated outcomes [9,10,11]. These severity strata are important in the context of inclusion criteria, guideline algorithms and trial comparisons. They can also be used in clinical trials to evaluate meaningful changes or to define endpoints [9]. Furthermore, validated PGA scores can be useful in the clinic for providing basic information about the patients’ profiles.

The validated Physician Global Assessment (PGA) scores for activity (worsening) and repigmentation (improvement) were used to determine how much change in affected body surface area (=VESplus) and in the number of progressing body locations (=VDAS and VDIS) is required to have mild, moderate or much change. So far, no severity strata for disease activity (worsening) and repigmentation (improvement) in vitiligo have been defined, whereas PGA scores are widely used in studies with no research-based thresholds [9,10,11]. In the current study, any change (slightest observed) in the affected BSA percentage resulted in a PGA ‘slight change’ score. Based on this study population, moderate PGA activity (worsening) begins when the vitiligo area has progressed by 27.7%, while repigmentation of 4.9% of the vitiligo lesions is sufficient for moderate PGA repigmentation (improvement). This marked difference seemed due to the pronounced influence of how many of the involved areas are repigmenting rather than the actual size of the repigmented area. Nonetheless, because the number of involved areas at baseline was limited in this study population (mean: 5.89; median 6) and the total body surface area was, in general, low (median (mean) VESplus: 1.3% (3.2%)), the cut-off points based on the PGA activity (worsening) and repigmentation (improvement) should be repeated in a patient population treated with for instance phototherapy and with more widespread disease.

The proposed thresholds aid in the interpretation of clinical trial results expressed in surface area/extent (BSA). In terms of the number of changed body areas, as a general rule, >1 repigmented location or >2 active body areas are associated with a moderate PGA score. This stratification should not be confused with the minimal important change (MIC) because the MIC requires the perspective of the patient (or physician) to determine the smallest change in an outcome that is important enough to make a change in treatment worthwhile [12]. Additionally, the instruction for the raters was provided to rank any (even the most tiny) change as ‘slightly’ worsened or improved. An important finding in this study is that the PGA scores were not only dependent on the percentage of change in extent but also on the number of changing body areas (=VDAS₁₅ and VDIS₁₅) and, in the case of repigmentation (=VDIS), also on the number of involved body areas at baseline.

A limitation of this study was the single-academic-center design including most often photo skin types II-III with rather limited BSA involvement, which may limit generalizability. There may also be significant differences in severity strata between different subpopulations, such as age groups (young children versus adults). Furthermore, data on other measurement properties such as responsiveness and cross-cultural validity are still required. A comparison of physicians’ and patients’ perspectives on the interpretation of activity (worsening) and repigmentation (improvement) would also be useful.

In conclusion, the current study confirmed the construct validity and reliability of the PGA for activity (worsening) and repigmentation (improvement), as well as identified potential strata for the VESplus, VDAS_15&60 and VDIS_15&60. These strata can be used to interpret numerical results generated by the VESplus, VDAS_15&60 and VDIS_15&60 and can guide treatment decisions in vitiligo management.