**3. Discussion**

Despite the increasing incidence of WNV in Europe and the Mediterranean area, few external quality assessments have been organized to evaluate the diagnostic capacities of labs, especially in the veterinary sector. In the human health sector, five EQAs were organized by the ENIVD and EVD-LabNet networks between 2006 and 2017 to evaluate the WNV diagnostic performance of human virology labs in different countries, mostly in Europe but also in Middle East and America [23–27]. In Italy, two EQAs were organized in 2010 and 2011 by the National Institute of Health to assess the capacity of the national blood transfusion centers to detect WNV genome on blood donations [28]. In the veterinary sector, only two inter-laboratory assays were carried out in 2010 and 2013 to evaluate the capacity of the National Reference labs for equine diseases in Europe and Morocco to detect WNV antibodies in horse sera [29].

The EQA we present here has a number of differential features with respect to prior WNV inter-laboratory trials. In the first place, the geographical coverage of the involved laboratories. In previous EQAs, most of the participant labs were European while in this case labs from 17 non-EU countries including North Africa, Balkans, Black Sea and Middle East regions have been involved. Of these, only one lab in Bosnia and Herzegovina had participated in the previous serology EQA organized by ANSES in 2013 [29]. Secondly, this EQA was organized as a final evaluation after a three year training period where the participant labs attended several hands-on workshops aimed at improving their diagnostic performance. Thirdly, as the diagnostic assay used for molecular detection of WNV is a triplex RRT-PCR that allows the simultaneous detection of USUV, this EQA also provides relevant data about the capacities of the labs to identify this emerging flavivirus. Last, but not least, the labs were provided with all the materials required to perform the recommended diagnostic assays. The objective was to facilitate as much as possible the participation of all beneficiary labs. Moreover, and to promote sustainability, positive extraction and PCR controls were also provided to be used as quality controls during this EQA but also to serve as reference material for the future diagnostic activities of the labs. Such material is otherwise difficult to obtain and was greatly appreciated by the participants.

An added value of this EQA is also the combination of molecular and serological methods since this integrated approach is essential for WNV surveillance in animal populations. Although previous EQAs had analyzed the performance of vet labs to detect WNV antibodies, this is the first international proficiency test to evaluate the molecular diagnostic capacities of animal diagnostic labs.

The panel for WNV genome detection consisted of 10 samples containing various concentrations of four different flavivirus strains: two WNV European strains representing lineages 1 and 2, one European USUV strain and the reference JEV Nakayama strain. Unlike previous WNV EQAs where the viruses were diluted in human plasma [23,27,28] or virus culture medium [24], in this case the viruses were spiked in different organs' homogenates, serum or blood from horses and birds to mimic as much as possible the clinical samples that the vet labs would analyze during real surveillance and outbreak investigations. Considering the current situation in many Mediterranean countries where different lineages of WNV co-circulate with other flaviviruses, two samples containing both WNV and USUV were included in the panel to evaluate the ability of the labs to identify coinfections.

The panel for antibody detection consisted of 10 sera from WNV infected horses with different IgG and IgM antibody titers. Overall, both panels conformed a very comprehensive proficiency test that allowed accurate evaluation of the WNV and USUV diagnostic capacities of the labs.

Some limitations were observed in the generic detection of flaviviruses. On the one hand, four laboratories (23.5%) did not perform this PCR. Apparently, in these labs real-time PCR has completely replaced conventional PCR and they even lack the necessary equipment to carry out the electrophoresis. This fact can have a negative impact in their diagnostic capacities for certain pathogens. In this case, they were unable to identify the presence of a non-WNV/USUV flavivirus in sample W9 (JEV). On the other hand, out of the 13 labs that performed the pan-flavivirus conventional RT-PCR, only 46.1% reported correct results in the whole panel, identifying the eight samples containing WNV, USUV or JEV. Most of the mistakes were false negative results, particularly in the three samples with lower viral load, for which weak positive bands were expected.

The performance of the labs with the triplex RRT-PCR was highly satisfactory in general terms, with the exception of lab #1 that reported incorrect results in all the samples. It seems that an error in the numbering of the tubes or during the transcription of the results could occur, as the reported result for each sample corresponded to the expected outcome of the preceding sample. Excluding this lab, the overall results for the specific detection of WNV L1 were excellent. The positive samples were identified by all labs except one false negative result in one lab, which represents an overall detection rate of 97.9%, much higher than that reported in previous EQAs organized by ENIVD and EVD LabNet networks in human virology labs [27,30]. With regard to the diagnosis of L2 WNV infections, these previous interlaboratory assays had evidenced important limitations in the participant labs. In the EQA organized in 2006 only 46.6% of the labs were able to detect L2 positive samples [23]. Although this percentage increased during the second ENIVD EQA in 2011, one third of the labs still failed to identify WNV L2 [24]. In the last EQA organized in 2017, the human labs had considerably improved the detection of WNV L2 but some false negative results were still reported [27]. In the present study, all the labs (except lab #1) successfully detected the presence of L2 in the positive samples of the panel. In the current epidemiological context, where L2 is already present in many Euro-Mediterranean countries and will probably expand to new territories [31,32], these results are highly relevant to ensure a timely detection of L2 circulation in animal populations of the involved countries.

As regards USUV detection, before this EQA, only two ring trials for human labs had included one USUV sample in the diagnostic panels [27,28]. Interestingly, in the EQA organized for blood transfusion centers in Italy in 2011, the USUV sample was misidentified as WNV positive by all the labs, indicating the presence of cross-reactivity in the two automated nucleic acid assays used [28]. Cross-reactions between both flaviviruses have been evidenced in several studies where automated commercial PCR kits were used to test human blood [33,34]. However, in the present EQA, USUV positive sample was reported as WNV negative by all the labs. This, together with the fact that the JEV sample was negative for the three viruses in all the labs, confirms the high specificity of the applied triplex PCR assay.

In the more recent EQA organized by EVD LabNet, the USUV sample was correctly detected by a small percentage of labs, even in countries with demonstrated USUV circulation, revealing a clear need for technical improvement in USUV diagnosis in participating labs [27]. In our EQA, 75% of the labs were able to identify the three positive samples and 81.2% detected two of them. Even so, two labs (11.7% of the total) could not detect the virus using the recommended PCR protocol due to the lack of the required fluorescent channel (Cy5) in their real-time thermocyclers.

The results of the generic antibody detection exercise were excellent, with 100% correct results in all labs except for one negative sample that one lab reported as doubtful. One limitation of this EQA is the lack of sera with antibodies directed against other flaviviruses, especially considering the high degree of cross-reactions that occur in serological assays [16]. Sera from USUV or JEV infected animals in su fficient amount to prepare an EQA panel are di fficult to obtain unless they originate from experimentally infected horses. Unfortunately, we could not have access to this type of samples and, therefore, the panel was restricted to WNV positive and negative sera. As a result, we can only evaluate the capacity of the labs to successfully identify WNV antibodies, but we cannot assess the potential interference of cross-reactivity with other flaviviruses on the performance of the ELISA kits.

More di fficulties were observed with the specific detection of IgM antibodies. This was most probably due to the fact that two of the 4 IgM positive sera had low IgM antibody titers with OD values close to the reference threshold. In fact, most of the incorrect results were false negatives in these two sera, while no false positive results were reported. The three labs that used an alternative kit (IDVet IgM capture ELISA) to evaluate the presence of IgM WNV antibodies were only able to detect the sample with the highest IgM titer. These results are in agreemen<sup>t</sup> with the data derived from the

serology EQA organized by ANSES in 2013, where the INgezim WNV IgM ELISA displayed higher analytical sensitivity than the IDVet IgM capture ELISA for L1 and L2 WNV infected horses [29].

At the end of the exercise, each participant laboratory received an individual report with the analysis of their results, possible reasons for the observed deviations and recommendations to improve their competence.

The unique characteristics of this EQA, where all the participant labs used the same protocols and reagents, enables a comprehensive evaluation of the selected diagnostic methods under "controlled" conditions. In this way, we could verify that the triplex RRT-PCR protocol [15] is a reliable method for accurate detection and di fferentiation of WNV L1, L2 and USUV in animal samples of di fferent type and origin. This protocol was easily transferred to all the labs that, in most cases, were able to correctly apply the assay and interpret the results. In the current epidemiological context, this method can be a very useful tool for clinical diagnostic and epidemiological surveillance of WNV and USUV. With respect to the pan-flavivirus conventional RT-PCR, more variability of results was observed, especially in samples with low viral loads. Three false positive results were also reported that could be due to the presence of unspecific bands or cross-contamination. However, this broad-range flavivirus assay is a good first-line tool for rapid flavivirus detection and a useful complement to species specific assays. Moreover, this technique allows further genome sequencing to identify the involved virus and perform phylogenetic analysis [35].

In the case of the antibody detection exercise, our results confirm the optimal performance of the Ingenasa competitive ELISA (INgezim West Nile Compac), with high reproducibility values, as all the labs reported concordant results. This commercial kit is widely used for WNV surveillance in birds and horses due to its excellent sensitivity and specificity values [16,29]. The recommended IgM ELISA (INgezim WNV IgM ELISA) also displayed good results and was able to identify positive samples that were not detected with other commercial kits.

This exercise o ffers a good overview of the WNV and USUV diagnostic capacities of veterinary labs in 17 EU-neighboring countries. Based on the obtained results, most of the participant labs have the necessary infrastructure and expertise to correctly perform the molecular and serological diagnosis of WNV. The training strategy developed during the MediLabSecure project, with two workshops (molecular and serological diagnosis) followed by this EQA, was beneficial in improving the capacities of the labs.

### **4. Materials and Methods**

### *4.1. Call for Participation*

An invitation letter was sent by the coordinating team of the MediLabSecure animal virology network (INIA-CISA, Madrid, Spain) to the beneficiary veterinary laboratories (*n* = 18). Seventeen laboratories accepted to participate (94.4%). The participation was free of charge and entailed the publication of comparative results in an anonymous manner.

### *4.2. Preparation of EQA Panel*

### 4.2.1. Samples for Virus Genome Detection

For the molecular diagnosis of WNV, each participant received a coded panel of 10 samples, as shown in Table 1.

Four viral strains were used for the preparation of the panel: SP07 strain (WNV L1), isolated from a golden eagle in Spain in 2007 [36]; AUS08 strain (WNV L2), isolated from a goshawk in Austria in 2008 [37], USU11 (Usutu virus) isolated from a blackbird in Italy in 2011 (GenBank number KX816649) [38] and the Nakayama strain (Japanese encephalitis virus-JEV). All the viral stocks were inactivated using ß-propiolactone. Absence of residual infectivity was confirmed after three consecutive passages in Vero cells by absence of cyto-phatic e ffect and by RRT-PCR analysis.

Several dilutions of inactivated viral stocks were spiked in di fferent matrices (serum, blood, liver, heart or kidney) from healthy non-infected birds and horses to prepare the positive samples. The negative samples consisted of brain and heart homogenates from healthy birds and horses. Nucleic acid extraction was performed from 200 μL of sample using the QIAamp ® Cador Pathogen Mini Kit (QIAGEN), following the manufacturer's instructions. In the final step, RNA was eluted in 50 μL of nuclease-free water. All samples were tested twice with two validated and widely used PCR techniques that we selected as recommended methods: a conventional RT-PCR for pan-flavivirus detection [35] and a RRT-PCR for simultaneous WNV and USUV detection [15].

For the conventional RT-PCR, mix was prepared in a final volume of 25 μL per sample containing 2 μL of RNA template, 0.6 μM of each primer (cFD2 and MAMD), RT-PCR enzyme mix and RT-PCR bu ffer of the commercial SuperScript ® III One-Step RT-PCR System with Platinum ® Taq DNA polymerase (Life Technologies, Thermo Fisher Scientific). All reactions were carried out using the following thermal profile: reverse transcription at 55 ◦C for 30 min, initial PCR activation step at 94 ◦C for 2 min, followed by 40 cycles of 30 s at 94 ◦C, 30 s at 55 ◦C, and 30 s at 68 ◦C and a final extension step of 5 min at 68 ◦C. Amplified products were analyzed by 2% agarose gel electrophoresis. Positive samples should give a specific band of the same size as the positive control (252 bp).

The RRT-PCR was performed using the primers, probes and the thermal profile described by del Amo et al. [15]. Samples with Ct > 40 were considered negative.

According to the obtained bands in the conventional RT-PCR and the Ct values in the triplex RRT-PCR, a collection of 10 samples was finally selected (Tables 1 and 2). The samples were aliquoted (1 mL) and each vial was lyophilized and stored at 4 ◦C until delivery to the participant laboratories.

Prior to delivery, the lyophilized panel was resuspended in DNAse-free water and was fully analyzed to verify the integrity of the samples and the reproducibility of the results after lyophilization. Triplicates of each lyophilized sample were analyzed by 3 technicians at INIA-CISA using the mentioned techniques. For the RRT-PCR, the reference Ct value was established as the mean of the nine repetitions (Table 2).

Two positive controls were delivered with the panel: (1) a triplex positive extraction control consisting of cell culture medium spiked with a mix of inactivated WNV L1, L2 and USUV strains to obtain, after a 1/10 dilution, an expected Ct value of 32 ± 2 for each virus (this sample was lyophilized and stored at 4 ◦C until delivery) and (2) a triplex positive reaction control consisting of a mix of WNV L1, L2 and USUV RNAs with an expected Ct value of 32 ± 2 for each virus (this sample was stored at −80 ◦C until delivery).

### 4.2.2. Samples for Antibody Detection

For the serological diagnosis of WNV, each participant received a panel of 10 samples (six positive and four negative) (Table 3). The positive samples included sera obtained from naturally infected horses in Southern Spain and one positive reference serum from the EU Reference Laboratory for equine diseases. Four of these samples were IgM positive, obtained from recently infected horses (Table 3). The negative samples consisted of one serum from a non-infected horse and a commercial negative horse serum (Biowhittaker). All samples were inactivated by heating at 56 ◦C for 45 min. Each sample was aliquoted (130 μL) and stored at −20 ◦C until delivery.
