Comparing and Correcting Spectral Sensitivities between Multispectral Microscopes: A Prerequisite to Clinical Implementation

Simple Summary

Multispectral, multiplex immunofluorescence (mIF) microscopy is an emerging technology for characterization of the tumour microenvironment. Achieving high-throughput collection and analysis of mIF microscopy images often requires the use of multiple microscopes, but it is not guaranteed that data from one microscope can be compared to data from another. We used a set of eight melanoma tissue samples to measure and correct for data differences between three microscopes. We scanned the samples twice on each microscope and measured the average tissue flux densities in the resulting sets of images. By applying a relatively simple calibration model accounting for sample- and microscope-specific effects, we were able to reduce the variations in raw image brightness and immune marker expression measurements by 79% and 72%, respectively. This shows that simple procedures can be used to effectively standardize mIF data from multiple microscopes for potential use in both research and clinical diagnostic settings.

Abstract

Multispectral, multiplex immunofluorescence (mIF) microscopy has been used to great effect in research to identify cellular co-expression profiles and spatial relationships within tissue, providing a myriad of diagnostic advantages. As these technologies mature, it is essential that image data from mIF microscopes is reproducible and standardizable across devices. We sought to characterize and correct differences in illumination intensity and spectral sensitivity between three multispectral microscopes. We scanned eight melanoma tissue samples twice on each microscope and calculated their average tissue region flux intensities. We found a baseline average standard deviation of 29.9% across all microscopes, scans, and samples, which was reduced to 13.9% after applying sample-specific corrections accounting for differences in the tissue shown on each slide. We used a basic calibration model to correct sample- and microscope-specific effects on overall brightness and relative brightness as a function of the image layer. We tested the generalizability of the calibration procedure and found that applying corrections to independent validation subsets of the samples reduced the variation to 2.9 ± 0.03%. Variations in the unmixed marker expressions were reduced from 15.8% to 4.4% by correcting the raw images to a single reference microscope. Our findings show that mIF microscopes can be standardized for use in clinical pathology laboratories using a relatively simple correction model.

1. Introduction

Multispectral, multiplex immunofluorescence (mIF) assays are emerging tools for biomarker discovery. They facilitate not only the study of basic cell population densities in a tissue-sparing manner, but also co-expression analyses, quantification of marker intensities, and spatial relationships. Multiple studies performed in numerous tumour types have demonstrated the predictive and prognostic benefit of being able to spatially resolve immunoactive cell populations within the tumour microenvironment (TME) and relate these findings to clinical outcomes [1,2,3,4,5,6,7,8,9,10,11,12,13]. In a meta-analysis of different biomarker modalities, mIF assays have been shown to have higher predictive value than tumour mutational burden, IFN-$\gamma$ gene signatures, and PD-L1 immunohistochemistry for predicting response to anti-PD-1-based therapies [14].

In the research setting, performing high-throughput processing and analysis of mIF samples could lead to faster biomarker discovery. In the clinical setting, rigorously validated mIF assays could enable individualized treatments with immune checkpoint inhibitors (ICI) for patients. Given the potential for mIF assays to be used in both research and clinical settings, it is imperative to ensure these assays are reproducible.

Currently, proof-of-principle studies [15] and guidelines [16] exist around demonstrating the reproducibility of the staining portion of mIF assays. There is still an unmet need for standardizing the microscopes themselves [17,18,19]. Here, we looked to extend reproducibility assessments to the multispectral microscopes necessary for scanning the mIF-stained tissue samples. Through the use of three different microscopes housed at a single academic institution, we were able to develop a relatively simple and deployable correction model capable of adjusting these multispectral microscopes to a single reference microscope.

2. Materials and Methods

Eight advanced formalin-fixed paraffin-embedded (FFPE) melanoma pathology specimens were obtained from the Johns Hopkins archives. Samples were de-identified and a 4 $\mathsf{\mu }\mathrm{m}$ section was cut from each block. Automated mIF was performed as previously described [9], but the mIF panel was expanded to include CD3 and a pan-membrane stain (Table S1).

Briefly, samples were baked offline for 3 h at 65 °C, then loaded onto the Leica BOND RX automated research stainer (Leica Biosystems, Buffalo Grove, IL, USA). Samples were then baked online at 60 °C for 30 min, and residual paraffin was removed (Dewax, Leica, Deer Park, IL, USA). Initial antigen retrieval was performed using a pH9 EDTA buffer (ER2, Leica) for 40 min at 100 °C. After initial blocking for endogenous peroxidases (BLOXALL, Vector Labs, Newark, CA, USA), non-specific antibody binding was blocked (Protein Block, Agilent, Santa Clara, CA, USA). Primary antibodies, polymers, and opals were applied for Position 1 (Table S1), then antibody stripping was performed using a pH6 sodium citrate buffer (ER1, Leica) for 20 min at 95 °C. This process was repeated for each position, after which slides were counterstained (Spectral DAPI, Akoya Biosciences, Marlborough, MA, USA) and wet mount coverslipped (Prolong Diamond, Invitrogen, Waltham, MA, USA). Prior to staining, the mIF panel was optimized to reduce cross-talk and/or bleed-through by performing primary, secondary, and fluorophore titrations to balance fluorophore intensities, as previously described [9,15]. All slides used were stained in the same batch so that batch-to-batch variations would not introduce additional artefacts.

The mIF-stained slides were scanned using PhenoImager HT (formerly known as Vectra Polaris) microscopes (Akoya Biosciences, Marlborough, MA, USA), which are automated multispectral microscopes capable of capturing fluorescent signals with wavelengths between 440 nm and 780 nm. These microscopes imaged samples by first passing light emitted from a multiband LED array through one of seven excitation filter cubes. This light illuminated areas of the tissue samples, stimulating the fluorophores and causing them to fluoresce. Fluorescence light was received by filter systems composed of seven static broadband filter cubes and 43 liquid crystal tunable narrowband filters. Light passing through each of the narrowband filters was captured by a CCD camera, forming a set of 43 monochromatic image planes. The 43-layer “raw” images were spectrally unmixed using the inForm software [20] (inForm^® v2.4.8, Akoya Biosciences, Marlborough, MA, USA), depending on libraries comprised of pure spectra for each fluorophore also captured on the PhenoImager HT microscopes. The spectral unmixing process transformed the raw images into 10 layers: one layer for each fluorophore, and an additional layer for autofluorescence. The 10 layers of the resulting “unmixed” images were analysed separately as measurements of individual marker expressions within the tissue samples [9,15].

Each slide was scanned twice on three different PhenoImager HT microscopes, for a total of six independent scans as summarized in Table S2. An independent scanning protocol was created for each microscope by auto-exposing on the brightest pixels for each broadband filter across the set of eight tissue samples. The broadband filters used to excite each fluorophore are listed in Table S1. Emission spectra for each fluorophore were captured across several broadband and narrowband filters, as shown in Figure S1. The microscope-dependent corrections discussed below were derived from, and applied to, the raw 43-layer multispectral images, so that a common library could be used to perform spectral unmixing on all data coming from different microscopes.

Tiling of the entire sample was achieved by acquiring 20% overlapping “high-power field” (HPF) image tiles, which were assembled into seamless whole-slide images as previously described [9]. On average, 5700 HPFs were acquired per round of scanning, totalling 34,725 HPFs across the entire dataset (Table S3). Each raw HPF image was stored as an array of unsigned 16-bit integers, with dimensions of 1872 × 1404 pixels and 43 layers. Each image layer contained the total brightness of each pixel in a specific, narrow range of light wavelengths. Image layers were grouped by the static broadband filters used to initially select wider ranges of wavelengths of light. The mIF narrow-band wavelengths contributing to each image layer and their corresponding broadband filters are plotted in Figure S1.

A binary image mask $B_h$ was generated for each raw HPF h, in which areas containing empty background or oversaturated pixels were set to 0 and areas showing well-imaged tissue were set to 1. Background pixels were determined using Otsu’s thresholding algorithm implemented in OpenCV [21]; oversaturated pixels were masked out using hand-tuned layer-dependent thresholds.

The raw HPFs were normalized by their exposure times in each image layer to produce the images $I_h$, with units of counts/ms. The “mean image” M for each scan of each sample was then calculated as

$$M=\frac{{\sum }_h{B}_h{I}_h}{{\sum }_h{B}_h},$$

(1)

describing the average flux of the tissue at each pixel in counts/ms.

These mean images were averaged over the two-dimensional pixel indices i and j in each image layer k to produce the set of ${\overline{X}}_{mrk}^n$ spectra,

$${\overline{X}}_{mrk}^n=\frac{1}{HW}\sum _{i,j}{M}_{mrk}^n,$$

(2)

describing the average flux of the tissue in each image layer k observed for scan r of sample n on microscope m, where H and W are the height and width of each image.

The ${\overline{X}}_{mrk}^n$ spectra were then normalized so that they impacted measurements relatively equally regardless of their individual brightnesses. First, the new spectra ${\tilde{X}}_{mrk}^n$ were calculated by multiplying each ${\overline{X}}_{mrk}^n$ by the fraction of the total sample coming from its own HPFs,

$${\tilde{X}}_{mrk}^n={\overline{X}}_{mrk}^n\left(\frac{h_{mr}^n}{{\sum }_{n,m,r}{h}_{mr}^n}\right),$$

(3)

and then the ${\tilde{X}}_{mrk}^n$ spectra were divided by their average over the $N=8$ samples, $M=3$ microscopes, $R=2$ scans, and $K=43$ raw image layers to produce the $X_{mrk}^n$ spectra,

$$X_{mrk}^n=\frac{{\tilde{X}}_{mrk}^n}{\frac{1}{NMRK}{\sum }_{n,m,r,k}{\tilde{X}}_{mrk}^n},$$

(4)

which represent the relative tissue flux variations about one for each microscope, scan, and sample as a function of multispectral image layer.

3. Results

The $X_{mrk}^n$ spectra are pictured in Figure 1, showing an initial variation in overall illumination and relative spectral intensity characterized by a standard deviation of 29.85% on average over all image layers. We used these spectra to develop a method of accounting for those differences, independently modelling contributions from the individual tissue samples themselves and from the three different microscopes.

To simplify calculations, a final set of factors $a_{mr}^n$ were calculated, representing the normalized average relative intensities of the samples per microscope per scan without any wavelength dependence:

$$a_{mr}^n=\frac{\frac{1}{K}{\sum }_k{X}_{mrk}^n}{\frac{1}{NMRK}{\sum }_{n,m,r,k}{X}_{mrk}^n}.$$

(5)

We first used a simple calibration model applied to the entire set of samples, which showed the reduction in variation that was possible to achieve overall. From this form of the calibrations, we propose a source of the observed differences between the individual multispectral microscopes. We then modified the model slightly to factor out the contributions coming only from the differences in the microscopes, and used a bootstrapping procedure to show that those microscope correction factors could be expected to generalize to additional samples. Finally, we performed three different spectral unmixings on the raw image data to evaluate how standardizing images to a single microscope affects marker intensity measurements, rather than raw image fluxes.

3.1. Correcting the Entire Dataset

Our goal in correcting the entire dataset overall was to effect the greatest possible reduction in the variance shown in Figure 1. We first removed tissue sample- and microscope-dependent differences in the overall brightnesses of each set of images, and then accounted for differences in the relative spectral sensitivities exhibited by each tissue sample and each microscope. An overview of the method is shown in Figure 2.

We began by applying two corrections to the overall brightnesses as functions of the tissue samples, $B_n$, and microscopes, $C_m$, calculated as

$$B_n=\frac{1}{MR}\sum _{m,r}{a}_{mr}^n\mathrm{and}{C}_m=\frac{1}{NR}\sum _{n,r}{a}_{mr}^n.$$

(6)

Applying these amplitude corrections resulted in the set of $x_{mrk}^n=X_{mrk}^n/(B_n·C_m)$ spectra pictured in Figure 3. These amplitude corrections alone reduced the variance observed from 29.85% to 14.83% on average over all image layers.

We next modelled the effect of varying spectral sensitivities in the specific tissues mounted on each slide. The relative variations in the sample dimension as a function of image layer, $T_{nk}$, were calculated by averaging the $x_{mrk}^n$ spectra over all microscopes and scans and used to determine wavelength-dependent $b_{nk}$ factors,

$$b_{nk}=\frac{T_{nk}}{\frac{1}{NMR}{\sum }_{n,m,r}{x}_{mrk}^n}\mathrm{where}{T}_{nk}=\frac{1}{MR}\sum _{m,r}{x}_{mrk}^n.$$

(7)

The $T_{nk}$ variations and $b_{nk}$ correction factors are pictured in Figure S2.

Applying the $b_{nk}$ tissue profile corrections to $x_{mrk}^n$ gave the set of $y_{mrk}^n=x_{mrk}^n/b_{nk}$ spectra, pictured in Figure 4. The standard deviation of the $y_{mrk}^n$ spectra was 10.56% on average over all image layers.

The variations remaining in the $y_{mrk}^n$ spectra corresponded to the wavelength-dependent relative differences between the three microscopes. The microscope-relative variations $P_{mk}$ and correction factors $w_{mk}$ were calculated similarly to the tissue variation spectra and corrections,

$$w_{mk}=\frac{P_{mk}}{\frac{1}{NMR}{\sum }_{n,m,r}{y}_{mrk}^n}\mathrm{where}{P}_{mk}=\frac{1}{NR}\sum _{n,r}{y}_{mrk}^n.$$

(8)

The $P_{mk}$ variations and $w_{mk}$ correction factors are shown in Figure S3.

Applying the $w_{mk}$ factors resulted in the set of $z_{mrk}^n=y_{mrk}^n/w_{mk}$ spectra, pictured in Figure 5. The $z_{mrk}^n$ spectra exhibited a 2.70% standard deviation variation on average over all image layers.

The reduction in overall variation between all samples, microscopes, and scans is shown in Figure 6. The upper plot shows the standard deviation over all samples, microscopes, and scans as a function of the image layer at each stage of correction, and the lower plot shows the averages of these standard deviations over all image layers. An initial standard deviation of 29.85% was reduced to 2.70% after applying corrections accounting for differences between tissue samples and microscopes.

3.2. Contributions to Microscope-Specific Correction Factors

The $w_{mk}$ factors can be averaged over all layers to calculate $w_m^{\mathrm{ill}}=\frac{1}{K}{\sum }_k{w}_{mk}$, which should be approximately equal to 1 since the $C_m$ amplitude corrections were already applied in $y_{mrk}^n$. Dividing out these overall scales and averaging over the layers $k^{†}$ belonging to each broadband filter group produced the set of $w_{mk}^{\mathrm{BB}}$ factors,

$$w_{mk}^{\mathrm{BB}}=\frac{1}{K^{†}}\sum _{k^{†}}\frac{w_{mk}}{w_m^{\mathrm{ill}}},$$

(9)

which quantify the differences between microscopes that are attributable to inhomogeneities in those microscopes’ specific broadband filters. Lastly, the differences specific to the piezoelectrically tuned narrow-band filters ($w_{mk}^{\mathrm{NB}}$) were quantified by dividing out both the overall illumination and broadband filter contributions:

$$w_{mk}^{\mathrm{NB}}=\frac{w_{mk}}{w_m^{\mathrm{ill}}·w_{mk}^{\mathrm{BB}}}.$$

(10)

The $w_m^{\mathrm{ill}}$, $w_{mk}^{\mathrm{BB}}$, and $w_{mk}^{\mathrm{NB}}$ contributions to the total $w_{mk}$ correction factors are pictured in Figure S4. It is clear that most of the differences in relative spectral sensitivities between microscopes can be attributed to inhomogeneities in the microscopes’ broadband filter cubes. Each microscope was manufactured with its own static set of broadband filter cubes, and it is expected that the materials used for those filter cubes may differ between instruments from the time of manufacture. Our data indicate that those differences can be as large as 20% with respect to the means of all three microscopes for any given broadband filter group.

This same effect is also visible when calculating the overall covariance matrices of the $X_{mrk}^n$, $x_{mrk}^n$, $y_{mrk}^n$ and $z_{mrk}^n$ spectra in the image layer dimension. The image layer-projected covariance matrix of $X_{mrk}^n$, for example, ${\Sigma }_{k_1{k}_2}\left(X\right)$, can be calculated as

$${\Sigma }_{k_1{k}_2}\left(X\right)=\frac{1}{NMR}\sum _{n,m,r}{d}_{mr{k}_1}^n{d}_{mr{k}_2}^n,$$

(11)

where

$$d_{mrk}^n\left(X\right)=X_{mrk}^n-{\mu }_{mk}\left(X\right)\mathrm{and}{\mu }_{mk}\left(X\right)=\frac{1}{NR}\sum _{n,r}{X}_{mrk}^n.$$

(12)

These image layer-projected covariance matrices at each stage of correction ${\Sigma }_{k_1{k}_2}\left(X\right)$, ${\Sigma }_{k_1{k}_2}\left(x\right)$, ${\Sigma }_{k_1{k}_2}\left(y\right)$, and ${\Sigma }_{k_1{k}_2}\left(z\right)$ are shown in Figure S5. They show an overall reduction in the scale of the variance as successive corrections are applied, as well as a strong correlation between groups of layers imaged with the same broadband filter, and between image layers corresponding to similar narrow-band wavelengths, as pictured in Figure S1.

3.3. Generalizing Microscope Correction Factors

Having determined a model for using a group of multiple-imaged tissue samples to measure microscope-dependent correction factors, we next investigated how generalizable the procedure would be if applied to additional data that were not used to measure the corrections. To this end, we used a bootstrapping procedure to repeatedly calculate microscope-dependent correction factors using particular subsets of the eight tissue samples and then applying those corrections to orthogonal subsets of the tissue samples. This procedure is displayed in Figure 7.

The data used in this procedure were normalized as before ($X_{mrk}^n$) and divided by the same tissue sample-dependent amplitude corrections $B_n$ to produce the set of ${\chi }_{mrk}^n$ spectra,

$${\chi }_{mrk}^n=\frac{X_{mrk}^n}{B_n}.$$

(13)

These spectra defined new ${\beta }_{nk}$ factors,

$${\beta }_{nk}=\frac{\frac{1}{MR}{\sum }_{m,r}{\chi }_{mrk}^n}{\frac{1}{NMR}{\sum }_{n,m,r}{\chi }_{mrk}^n},$$

(14)

analogous to the $b_{nk}$ factors, to account for spectral variations attributable to differences between the tissue samples on each slide. Dividing by these factors produced the set of ${\psi }_{mrk}^n$ spectra,

$${\psi }_{mrk}^n=\frac{{\chi }_{mrk}^n}{{\beta }_{nk}},$$

(15)

in which any remaining variations were attributable to the different microscopes.

At each iteration s of the bootstrapping procedure, $N^{\prime }=5$ “fit” samples $n_s$ were randomly chosen from the full set of eight, and the microscope-dependent correction factors $C_m^s$ and ${\omega }_{mk}^s$ were calculated using just those five samples:

$$C_m^s=\frac{1}{N^{\prime }R}\sum _{n_s,r}{a}_{mr}^{n_s}\mathrm{and}{\omega }_{mk}^s=\frac{\frac{1}{N^{\prime }R}{\sum }_{n_s,r}{\psi }_{mrk}^{n_s}}{\frac{1}{N^{\prime }MR}{\sum }_{n_s,m,r}{\psi }_{mrk}^{n_s}}$$

(16)

The correction factors were then applied back onto these five samples to produce the $z_{mrk}^{n_s}$ spectra,

$$z_{mrk}^{n_s}=\frac{{\psi }_{mrk}^{n_s}}{C_m^s{\omega }_{mk}^s},$$

(17)

and also to the three orthogonal “test” tissue samples. The post-correction standard deviations across the fit and test samples, plus all microscopes and scans, were calculated. This procedure was repeated 56 times, once for each independent choice of the five fit samples.

Figure S6 shows the distribution of the $C_m^s{\omega }_{mk}^s$ factors calculated for each iteration; the microscope-dependent correction factors were all very similar regardless of the subset of samples used to calculate them. Figure 8 shows the standard deviations across all samples, microscopes, and scans of the original $X_{mrk}^n$ data, the tissue-homogenized ${\psi }_{mrk}^n$ spectra, and the fully-corrected $z_{mrk}^{n_s}$ spectra for all fit/test sample subsets. The $z_{mrk}^{n_s}$ data points shown are the averages over all bootstrapping iterations, with error bars equal to the standard deviation.

Applying microscope-dependent corrections calculated using orthogonal subsets of samples reduced the standard deviation from 13.87% to 2.91 ± 0.03% on average over all image layers, comparable to the final standard deviation of 2.66 ± 0.01% observed when applying corrections back onto the subsets of samples used to calculate them. This shows that normalizing and homogenizing a set of tissue samples using $B_n$ and ${\beta }_{nk}$ correction factors reliably leaves only microscope-dependent variations present, and that corrections for those microscope variations can be reliably applied to new tissue samples from the same microscopes.

3.4. Impact to Measurements of Marker Expressions

Immunofluorescence microscopy is often used to measure the expressions of multiple biomarkers simultaneously. The PD1/PDL1 immunofluorescence panel used to stain the tissue samples described in Section 2 contained stains targeting CD3, PDL1, FoxP3, CD8, PD1, CD163, and Sox10/S100 proteins, as well as a DAPI stain targeting cellular nuclear DNA, and a lab-developed combination (“pan-membrane”) stain targeting cellular membranes. The inForm Automated Image Analysis Software (inForm^® v2.4.8, Akoya Biosciences, Marlborough, MA, USA) [20] from Akoya Biosciences was used to “unmix” the raw, 43-layer images into new, 10-layer images depicting the normalized expressions of each marker plus a layer for autofluorescence. We then quantified the effects of applying corrections for differences between microscopes on those measurements of marker expressions.

The spectral unmixing process depends on “library” slides as input, which provide measurements of individual marker responses and autofluorescence at different wavelength ranges. We investigated three different unmixing scenarios, depicted in Figure 9.

First, we unmixed the raw data using a single library whose slides were imaged on microscope 2. Then we performed a second unmixing using three different libraries whose slides were imaged on each of the three microscopes, where raw data were unmixed using the library from the microscope on which they were scanned. In the final scenario, we first applied factors to standardize all images to measurements from microscope 2, and then unmixed all of the corrected images using the single library from microscope 2.

The standardization factors applied were the means of the $C_m^s$ and ${\omega }_{mk}^s$ factors shown in Figure S6, divided by the factors for microscope 2, so that microscope 2 data were left unaltered and data from microscopes 1 and 3 were standardized to that single reference. The standardization was performed by dividing each raw image by the product of the $C_m^{\prime }$ and ${\omega }_{mk}^{\prime }$ factors, as in Equation (17).

The three sets of unmixed images were multiplied by their binary image masks and their average brightnesses in each layer were calculated and normalized as in Equations (1)–(4) above, except that the number of image layers was $K^{\prime }=10$ instead of $K=43$. Tissue sample-specific normalization factors $B_n^{\prime }$ and ${\beta }_{n{k}^{\prime }}^{\prime }$ were calculated as in Equations (6) and (14), respectively, and applied as in Equation (15). The resulting three sets of ${\psi }_{mr{k}^{\prime }}^{\prime n}$ spectra, one for each unmixing method, are shown in Figure S7 (the autofluorescence layer is omitted). The standard deviations across all samples, microscopes, and scans of these spectra are shown in Figure 10, along with their averages over all but the autofluorescence layer.

The uncorrected images unmixed with the microscope 2 library showed a remaining variation characterized by an average standard deviation of 15.84%, slightly larger than that observed in the tissue sample-corrected $y_{mrk}^n$ and ${\psi }_{mrk}^n$ spectra in Figure 6 and Figure 8, respectively. The uncorrected images unmixed with the individual microscope libraries had an average standard deviation of 8.49%, showing that using microscope-specific libraries in unmixing does compensate for some, but not all, systematic differences between samples imaged on different microscopes. The microscope-corrected images unmixed using the microscope 2 library showed an average standard deviation of 4.39%, slightly larger than the fully corrected $z_{mrk}^n$ spectra in Figure 6 and Figure 8.

The greatest reduction in the unmixed images’ microscope-specific differences was therefore observed by standardizing the fluxes of the raw images to a single reference microscope, and then unmixing all images using a library from that single reference microscope. The variations remaining in the unmixed images were slightly larger than those remaining in the raw images; likely due to the dimension reduction from 43 to 10 image layers that is inherent to the unmixing process.

4. Discussion

The use of immune checkpoint inhibitors (ICI), has completely changed the landscape of treatment for patients with advanced melanoma and other tumour types [22]. Two recently published clinical trials treating naïve patients with advanced melanoma showed a five-year overall survival (OS) $>40$% in patients treated with anti-PD-1 [23,24]. Additionally, patients treated with a combination of anti-PD-1 and anti-CTLA-4 showed an even higher median 6.5-year OS compared to patients treated with anti-PD-1 alone [25]. A pre-treatment biomarker to help predict which patients are more likely to respond to therapy is of great interest. Currently, there are no FDA-approved companion diagnostics to determine if patients with advanced melanoma should receive ICI [26]. Initially, a PD-L1 immunohistochemistry assay was approved as a complementary diagnostic, but this was ultimately rescinded after levels of PD-L1 expression did not correlate with OS [27]. More recently, a 6-plex mIF assay for predicting objective response, progression-free survival, and OS for patients with advanced melanoma receiving anti-PD-1-based ICI was developed [9].

There are many steps involved with creating a companion diagnostic assay including, but not limited to, demonstrating high intra- and inter-observer reproducibility of the assay [28,29,30]. This includes demonstrating little variability between multiple reagent lots, validating all instruments involved with performing the assay, and potentially creating a “locked-down” analysis algorithm for those assays requiring image analysis. In collaboration with several other groups, we have performed the initial steps for validating and determining the reproducibility of a mIF 6-plex assay by showing a strong inter- and intra-site concordance of both cell population densities and marker intensity measurements [15]. Some limitations of this study were that only the reproducibility of the mIF staining itself was tested, and that only regions of interest within the mIF-stained slides were scanned and analysed. Here, we expanded the scanned image to include the whole slide and standardize the multispectral microscopes used to acquire the imagery.

Through the serial scanning of eight advanced melanoma FFPE mIF-stained sections we were able to characterize systematic differences between three PhenoImager HT microscopes and showed that these differences are due to inhomogeneities in the broadband filter cubes built into each microscope. We developed a simple correction model that shows measurements of microscopes-specific correction factors are relatively agnostic to the specific samples used to measure them. Additional work may be needed to determine if these factors remain agnostic when scanning is performed on tissue from other tumour types, as there can be significant differences in staining patterns and background autofluorescence between tumour types. The proposed correction model factors out differences in tissue area across samples and differences between microscopes, making it possible to standardize image data from multiple microscopes to the mean of all microscopes or to a single reference microscope. By standardizing these data to a single reference microscope, we were able to reduce microscope-dependent flux variation in raw images by 79%, and in marker expressions measured in the spectrally unmixed images by 72%. Microscope-specific corrections of this form could allow for the harmonization of mIF assay results across institutions. With such harmonization, it may be possible to use a single set of software phenotyping projects across all samples, which is a pre-requisite for the development of “locked-down” analysis algorithms. More work will be needed to measure and test corrections of this form for microscopes housed at different institutions, and any microscope-specific standardization procedures must remain independent of other standardization steps performed to ensure reproducibility of marker panels or other aspects of imaging. Our group is also developing a method to standardize image data from slides stained in multiple batches, which will be the subject of a forthcoming publication.

These investigations imply a procedure to allow high-throughput mIF imaging using more than one PhenoImager HT microscope. For example, if a large number of slides are obtained all at once for imaging, several slides can be reserved for imaging on all available microscopes to determine microscope-dependent correction factors, and the rest can be imaged on only one microscope. HPFs from microscopes other than the chosen reference microscope can be corrected by the measured standardization factors, and then unmixed using only one library imaged on the reference microscope. The results presented here are only applicable to the specific PhenoImager HT microscopes at a single academic institution. It is expected that other PhenoImager HT systems would exhibit comparable differences due to their own static broadband filter cubes, and that the same method of measuring and applying corrections before spectral unmixing using multiple-imaged samples of a single tissue type would be a reasonable method for quantifying those differences as realized within that tissue type. Additional factors would need to be considered in developing correction models for multispectral image data from other systems.

5. Conclusions

We have characterized the differences in tissue fluxes observed in mIF microscopy data collected using three different multispectral microscopes at JHU. We used a basic sequential calibration model to measure and apply sample- and microscope-specific effects on the overall brightness and relative brightness as a function of image layer/narrow-band wavelength. We investigated the effects of generalizing the calibration procedure to additional data using a bootstrapping method. It was observed that an initial standard deviation in the average tissue fluxes across all microscopes, scans, and samples of 29.85% on average over all image layers was reduced to 13.87% after applying sample-specific corrections accounting for differences in the tissue shown on each slide. Applying microscope-specific corrections to orthogonal sample subsets further reduced the variation to 2.91 ± 0.03%. Variation in marker expressions observed in corresponding spectrally unmixed images was reduced from 15.8% to 4.4% by correcting raw images to a single reference microscope before unmixing. Our findings show that mIF microscopes can be standardized for use in clinical pathology laboratories using a relatively simple correction model that can reduce variation between microscopes by 79% in raw images and 72% in spectrally unmixed images.