Regionally Adaptive Active Learning Framework for Nuclear Segmentation in Microscopy Image

Abstract

Recent innovations in tissue clearing and light-sheet microscopy allow the rapid acquisition of intact micron-resolution images in fluorescently labeled samples. Automated, accurate, and high-throughput nuclear segmentation methods are in high demand to quantify the number of cells and evaluate cell-type specific marker co-labeling. Complete quantification of cellular level differences in genetically manipulated animal models will allow localization of organ structural differences well beyond what has previously been accomplished through slice histology or MRI. This paper proposes a nuclei identification tool for accurate nuclear segmentation from tissue-cleared microscopy images by regionally adaptive active learning. We gradually improved high-level nuclei-to-nuclei contextual heuristics to determine a non-linear mapping from local image appearance to the segmentation label at the center of each local neighborhood. In addition, we propose an adaptive fine-tuning (FT) strategy to tackle the complex segmentation task of separating nuclei in close proximity, allowing for the precise quantification of structures where nuclei are often densely packed. Compared to the current nuclei segmentation methods, we have achieved more accurate and robust nuclear segmentation results in various complex scenarios.

1. Introduction

Cell biology occurs over an enormous range of scales, with critical events taking place at the atomic, molecular, and cellular levels [1]. Ideally, we would like to visualize all of the events that take place in a cell, since this would give the most complete picture of cell function and the complex networks of interactions that give rise to the observed phenotype and cellular behaviors. For example, our understanding of nervous system function and dysfunction is critically dependent on visualizing and quantifying the three-dimensional structure of the brain. Light microscopy together with tissue clearing can be applied to image intact brain samples at high throughput in order to quantify phenotypic differences [1,2,3]. The accurate detection, localization, and quantification of all cells across the entire brain will further our understanding of how genetic variation alters brain structure resulting in neuropsychiatric disorders [4]. Automatic nuclei or cell segmentation methods are in high demand in many neuroscience studies to calculate the number of cells of a given type [5].

However, none of the existing image segmentation algorithms address the complexity of nuclear segmentation within whole microscopy images (usually target local image patch, where it is not very urgent to automatically count cells), especially targeting the following challenges:

• Poor image quality. Image quality is limited by the acquisition microscope hardware and imaging time. As shown in Figure 1, intensity inhomogeneity (displayed in the red box) is a common issue in microscopy. It is difficult to find an optimal threshold to segment nuclei that works well for the entire microscopy image. Other difficulties include image noise and low image contrast.
• Densely packed nuclei. When nuclei are in close proximity, the boundaries between nuclei are difficult to delineate. As shown in Figure 1, the cells in the blue box are densely packed, making accurate segmentation more difficult. The situation is very common, such as in the dentate gyrus of brain hippocampus. Since the topology of densely packed nuclei is very complex in terms of the number of nuclei in close proximity, current state-of-the-art methods, which work well to segment a single nucleus, usually fail to separate the boundaries between multiple nuclei touching one another.
• Large image dimensionality. In general, the computational challenges are based on the data size and complexity. A typical mouse brain (volume of ∼1000 mm³) imaged at high-resolution (e.g., 0.4853 μm × 0.4853 μm × 1 μm) results in approximately 8 TB of data (16-bit) for each acquired channel. Thus, it is computationally prohibitive to use sophisticated image processing methods that require matrix inversion or fit the contour of nuclei in an iterative manner.

For the goal of nuclear segmentation within microscopy images, we refer to several effective methods in computer vision. In deep watershed transform [6], the complex learning task is broken down into several intermediate tasks and networks are trained step by step. In our proposed method, we follow a similar approach to gradually improve the segmentation in a cascaded manner. Our regionally adaptive nuclear segmentation strategy is conceptually similar to the focal loss function in RetinaNet [7] in terms of hierarchically treating the segmentation (or object detection) with respect to the confidence in the correct classification. Januszewski presented a flood-filling network [8] to iteratively refine the tracing of the neurons from electron microscopy data by adding a recurrent loop that can check the current predictions and fix the segmentation errors in the next networks. Reka et al. comprehensively reviewed the most recent single-cell segmentation tools [9].

To address these challenges, we propose a regionally adaptive active-learning solution for nuclei segmentation in tissue-cleared microscopy images. Specifically, a cascaded approach is used, where both low-level image appearance features and high-level nuclei-to-nuclei contextual information are used to gradually improve the nuclear segmentation result by refining the contextual features based upon a more and more accurate nuclei probability map. Meanwhile, we propose a cascading architecture with a cross-training scheme to resolve the error accumulation problem in the cascading architecture. In addition, we have further developed a regionally adaptive fine-tuning strategy to actively learn the most challenging situation where multiple touching nuclei are in close proximity with blurry boundaries.

2. Related Work on Nuclei Segmentation

Various computational methods have been proposed in the last decade to automatically segment nuclei from microscopy images. In brief, the existing nuclei/cell segmentation algorithms can be classified into the following categories, as described below:

• Classic image processing techniques. The majority of current nucleus/cell segmentation methods are based on several classic image processing algorithms such as intensity thresholding [10,11,12,13,14], morphology operations [15,16,17], region accumulation [18,19,20,21], and deformable models [22,23]. Intensity thresholding might be the simplest and computationally cheapest method for cell segmentation. However, intensity thresholding, even the local thresholding technique such as local Otsu [24], does not lead to accurate segmentations within regions of densely packed nuclei since the boundaries are usually blurry. Similarly, a multi-level segmentation method using Darwinian particle swarm optimization (DPSO) was proposed in [25], which works in the frequency domain and assumes the nuclei voxels usually have a high response to certain specifically designed band-pass filters. The region-growing technique starts from a set of selected seed points in the image and iteratively adds connected points to form the labeled regions [5]. The significant limitation of the region-growing and watershed transform is its propensity to over-segment, due to image noise. To address this limitation, marker-based watershed methods have been proposed [18,26] to improve the segmentation accuracy by using the detected nuclei as the seeds to guide the segmentation of nuclei. However, it is very challenging to determine the localization of markers in large-scale microscopy images [27]. Recently, a cell detection method called CUBIC was proposed in [3] to construct a single-cell resolution whole brain atlas. CUBIC is a hybrid 2D/3D method where a 2D mean filter is first used to detect low maximum intensity values at the center of the nuclei and then a 3D filter is applied to unify the multiply-detected cells. However, CUBIC is unable to provide morphological information such as size and shape of nuclei, which might limit its application.
• Nuclei segmentation using unsupervised learning. K-means clustering has been applied to obtain an initial segmentation in [28,29], followed by a set of post-processing steps to refine the segmentation results. Probability inference under the EM (expectation maximization) framework is also used in automated nuclei segmentation for various biomedical data [30,31,32]. In [33,34], spectral clustering is used to identify the boundary of a cell, where the spatial and color information are used to measure similarity between the neighboring points. Furthermore, Song et al. proposed an improved nuclei segmentation method using the adaptive hierarchical clustering algorithm [35]. To improve the robustness of nuclei segmentation results, Su et al. [36] proposed a semi-automatic segmentation method that uses super-pixels to determine the segmentation label for each super-pixel, instead of each point. These unsupervised approaches, which are globally optimized for all the points in the entire image, are often not robust against the presence of substantial intensity inhomogeneity.
• Nuclei segmentation using supervised learning techniques. A coarse-to-fine nucleus segmentation framework was proposed in [37], where a multiscale convolutional network (MSCN) was integrated with the graph partition-based method for cytoplasm and nucleus segmentation in cervical images. Ronneberger et al. proposed the U-net architecture and applied the U-net to a cell segmentation task in light microscopic images [38]. Attention U-net introduces the attention and residual modules in the U-Net for the segmentation of live cells in microscopy images [39]. Cellpose proposed a generalist for a wide range of images of cell-based U-net [40]. Mask-RCNN used object detection technologies to achieve instance segmentation of nuclei [41]. ClearMap 1.0 [2] is a widely used computational software in the neuroscience community that provides functions such as cell detection, localization, and quantification of nuclei. The ilastik [42] is the component in ClearMap for nuclear segmentation, which uses random forest techniques [43].

For comparison, we show the characteristics of the most representative methods and our proposed method, named a regionally adaptive network via cascading architecture (RANCA), in

Table 1. Review of all nuclear segmentation methods under comparison.

	Touching	Morphology	Adaption	w/o Post-Processing
ilastik	×	✓	✓	✓
U-net	×	✓	✓	✓
Attention U-net	×	✓	✓	✓
Mask-RCNN	✓	✓	✓	✓
Cellpose	✓	✓	✓	×
RANCA	✓	✓	✓	✓

, in terms of the following: (1) Ability to tackle touching nuclei in densely packed region; (2) Ability to quantify morphological information (such as nuclei size and shape) for each individual nucleus; (3) Adaption to local structure (global or local approach); (4) No need for post-processing touching nuclei, where × and ✓ indicate the absence and presence of these capabilities, respectively.

Our RANCA method is designed for accurate nuclear segmentation in microscopy images. After a comprehensive evaluation of the varieties of microscopy images, we have achieved more accurate and robust segmentation results than the currently available methods, indicating the promising potential of our proposed nuclei segmentation method to quantify cell-level structure differences in tissue-cleared organ samples.

3. Methods

Inspired by the hierarchical information processing system of the human visual cortex, an active-learning cascaded model is presented in this paper. In general, there are two main components in our neural network architecture: (1) a cascading architecture with a cross-training scheme for nuclei segmentation that utilizes both low-level appearance features and high-level contextual features; and (2) a fine-tuning network to adaptively refine the segmentation result for a nucleus well-separated from its neighbors and densely packed nuclei, respectively. We will first present the cascading architecture with a cross-training scheme in Section 3.1, and then present our adaptive fine-tuning strategy for nuclei in close proximity in Section 3.2. We summarize our nuclear segmentation method in Section 3.3.

3.1. Cascading Architecture with Cross-Training

Due to low image contrast, noise, and intensity inhomogeneity, low-level features derived from image intensity are often not sufficient to guide the training of neural networks. Other high-level features are necessary to improve segmentation in cases of poor image quality. The nuclei segmentation task is broken down into several intermediate tasks and networks are trained step by step. The output of the initial nuclei segmentation task includes the probability of nuclei at each pixel. The network can extract patch-wise contextual features [44,45] based on the tentative nuclei probability map, which encodes the spatial relationship between surrounding nuclei. Leveraging these high-level heuristics, we can enhance the reliability of the nuclei probability map by training another network using both appearance and contextual features of different training data, and then use the refined contextual features to initialize the next network and so on until the segmentation results converge. However, when the model generates incorrect predictions with high confidence, these incorrect predictions will be further used as input to train the next cascade network. It is difficult for the model itself to rectify these false predictions as that scheme becomes less accurate. To alleviate this problem, we use cross-training, which trains two streams of models (Model i∼a and Model i∼b) simultaneously, as shown in Figure 2. These two independent models of concurrent training have the same structure but use different training data; the details about the network architecture are in Supplementary Materials. These two models help each other to rectify the false predictions. Specifically, Model i∼b is trained using the training data a to produce a nuclei probability map $P_{i_{\sim }b}$, and then $P_{i_{\sim }b}$ is treated as another feature channel, which is combined with image information I to jointly train Model i+1∼a. In Figure 2, the solid box and dotted box represent the training and inference processes, respectively.

The single model is the same as DeepCell [46], which produces initial nuclei segmentation and progressively improves the nuclei segmentation results by using recurrent connections. The output of the model is the probability of the underlying center point being a nucleus, nucleus edge, or background. The larger the probability value is, the more likely the center of the input patch belongs to the corresponding category. In the cascading architecture, the input to the first model (Model 1∼a/b) only consists of image intensity patches. After we apply the first model to all image points of training samples, we can obtain an initial probability map of nuclei that conveys high-level nuclei-to-nuclei information. The nuclei probability map produced by the previous model is treated as another feature channel. Compared to a single model, the convolutional procedure in the first convolutional block becomes a two-channel one. The nuclei probability map $P_{i_{\sim }a/b}$ produced by the previous model is combined with image information to check the current predictions and fix the segmentation errors in the next networks. The cascading architecture enhances the discriminative power of the model based on the improved nuclei probability map. Thus, by repeating the same cascading procedure we can progressively improve the segmentation result by leveraging more and more discriminative contextual features that are derived from the more and more accurate nuclei probability maps.

3.2. Regionally Adaptive Fine-Tuning Strategy

The cascading architecture with the cross-training scheme presented in Section 3.1 does not consider the spatial distribution of nuclei. The density of nuclei is very high in some regions, such as the dentate gyrus of the hippocampus, where accurately segmenting densely packed nuclei becomes very difficult. Since boundaries of touching nuclei in a densely packed region often have relatively low contrast compared with well-separated nuclei, we propose to treat these two nuclear segmentation scenarios differently in order to achieve active learning from selected samples. Specifically, we propose a regionally adaptive fine-tuning strategy to train a specific model with the most relevant annotated training samples for identifying nuclei boundaries when nuclei are in close proximity and appear in the image as touching, as demonstrated in Figure 3. To do so, we need to have (1) a detection method to mark the regions having unconnected nuclei and regions with touching nuclei, (2) a specific model to identify boundaries for touching nuclei in densely-packed regions.

(1) Detect touching nuclei. First, we apply the learned cascaded models to the existing training samples. Then, we binarize the nuclei probability map by using a thresholding technique (set to 0.5), obtaining the approximate mask of nuclei in the microscopy images. Next, we examine the shape of every connected component in the binary image to classify single nuclei and clumped nuclei. Ovality, concavity, and area of connected regions are used as the criteria to distinguish the unconnected nuclei from the touching nuclei.

• Ovality is measured by the ratio of the major axis and minor axis of fitting an ellipse (red ellipse in Figure 3a). For most individual nuclei, the contour is close to a circle. We can identify the region of suspected nuclei clumps if the ratio is greater than a threshold (here we use 3 for our image data).
• Concavity is measured by the difference in size between convex hulls (red curve in Figure 3b) and connected regions (green curve in Figure 3b). If the concavity degree is greater than a certain threshold (here we use 30 for our image data), it is highly possible that the underlying region is a clump of nuclei.
• The area of connected region. As shown in Figure 3c, it is straightforward to detect a region containing densely packed nuclei based on the area of the connected region being larger than a common nucleus area (here we used 500 pixels for our image data as threshold).

An example of the detected touching nuclei is shown in Figure 4, where the regions with touching nuclei are marked in blue.

(2) Active learning from selected samples. Given a separated probability map and the carefully delineated manual annotations of each nucleus, the network training procedure is identical to the previous networks. This specific model for touching nuclei focuses on the blurry boundaries between nuclei in close proximity. The advantage of adaptively considering single nuclei and multiple touching nuclei is shown in Figure 5b,c. As the intensity profile along the blue line shown on the right, it is much easier to separate two touching cells (designated by red arrows) after applying the specific models (Figure 5c) than only using the cascaded models (Figure 5b). It is possible that oblong endothelial nuclei could be mistakenly detected as touching nuclei because they match the first criterion in the detection module. However, it is less likely that our specific models for densely packed nuclei would split the single oblong nucleus since the intensity profile inside a single oblong nucleus (relatively bright in the middle) is completely different as compared to the appearance pattern of touching nuclei (relatively dark in the middle).

3.3. Complete Workflow of Regionally Adaptive Network via Cascading Architecture (RANCA)

Considering the computational cost, we cascaded three layers of models for segmenting nuclei from background. In our study, in the fine-tuning scheme, one specific model is used for well-separated nuclei and three specific models for densely packed nuclei. To segment nuclei in a tissue-cleared microscopy image, the following steps are performed sequentially.

1.

Apply the cascaded models at each point of a microscopy image patch, where the output of a layer is the average of output from the concurrent training models. The input is the image intensity image patch (Figure 6a). The image intensities are normalized by the mean of image pixel values. The output is the probability of being background, edge, and nuclei for the center point. After that, we can obtain the probability map for background, nuclei, and nuclei edge, as shown in Figure 6b;

2.

Apply the nuclei clump detection approach to partition whole images into regions with multiple touching nuclei and unconnected nuclei, respectively.

3.

Apply the specific models on the separated probability maps of touching and single nuclei, respectively. The outputs are shown in Figure 6c and Figure 6d, respectively.

4.

Combine (by adding) the segmentation results to obtain the final segmentation, as shown in Figure 6e.

The final segmentation includes nuclei boundaries and masks. Hereafter, we use CAC to denote the nuclear segmentation method only using a cascading architecture with cross-training, as well as RANCA to denote our complete nuclear segmentation method using a cascading architecture with cross-training and our regionally adaptive fine-tuning strategy.

4. Experiments and Results Analysis

4.1. Experimental Setup

The data we used are mainly from the image set BBB038, available from the Broad Bioimage Benchmark Collection [47]. A sample image is shown in Figure 1. We selected 42 patches in the dataset and manually labeled more than 9000 nuclei in 2D images. In the training stage, we selected training samples from 16 manually annotated patches (others are for testing). Our training data are increased by rotation and reflection of the original image patches to a total of over 156,000 training patches (71 × 71 pixels), each patch with the known label (nucleus, edge, and background) at the patch center. The purpose of augmenting training data by rotation and reflection is to obtain robust features for various nuclear morphologies. Max-pooling of a 2 × 2 window is operated to combine the activation vectors from convolutional filters. All weights in models were learned using stochastic gradient descent. The learning rate is initialized as 0.01 and decayed by 0.005%, and the momentum is set to 0.9.

Besides visual inspection with respect to the manual annotations, we further calculate the degree of spatial overlap between the manual annotations and the automatic nuclei segmentation results, which include Dice, Jaccard coefficient, Dice false positive (Dice FP), and Dice false negative (Dice FN), as well as nuclei counting. The overlap degree is measured by the Dice and Jaccard coefficients, which are widely used to evaluate segmentation accuracy. The Dice and Jaccard coefficients are defined as:

$$\mathrm{Dice}(R,S)={\displaystyle \frac{2|R\cap S|}{\left|R\right|+\left|S\right|}}$$

(1)

$$\mathrm{Jaccard}(R,S)={\displaystyle \frac{|R\cap S|}{|R\cup S|}}$$

(2)

where R and S are manual segmentation and binary segmentation results by an automatic nuclei segmentation method, respectively. In addition, Dice false positive (Dice FP) and Dice false negative (Dice FN) are also used to evaluate the specificity of segmentation performance, where the Dice FP quantifies the error ratio of mis-segmenting background as nuclei and Dice FN quantifies an error ratio of missed nuclei, respectively. The Dice FP and Dice FN are defined as:

$$\mathrm{Dice}\mathrm{FP}(R,S)={\displaystyle \frac{2|\overline{R}\cap S|}{\left|R\right|+\left|S\right|}}$$

(3)

$$\mathrm{Dice}\mathrm{FN}(R,S)={\displaystyle \frac{2|R\cap \overline{S}|}{\left|R\right|+\left|S\right|}}$$

(4)

where $\overline{R}$ and $\overline{S}$ stand for the complements of R and S, respectively.

4.2. Evaluation of Segmentation Performance

4.2.1. Comparion with Other Methods

We evaluate the segmentation results in various challenges in nuclei segmentation by comparing with other methods. The comparison to other counterpart methods include local Otsu’s method using region adaptive threshold, referred to as (1) ilastik [42], (2) U-net [38], (3) Attention U-net [39], (4) Mask-RCNN [41], (5) Cellpose, and (6) our complete nuclear segmentation method (referred to as RANCA), respectively.

For analyzing the effectiveness and robustness of methods, we evaluated the segmentation results for representative nuclei segmentation challenges. In the top of Figure 7, we show five representative segmentation challenge cases, including (a) low image contrast, (b) intensity inhomogeneity, (c) touching nuclei, (d) intensity inhomogeneity and low image contrast, and (e) touching nuclei with intensity inhomogeneity. The corresponding segmentation results by RANCA are shown in the bottom of Figure 7, where red contours denote automatically predicted nuclei boundaries.

Table 2. The segmentation results of nulei images. The segmentation accuracy is compared with other nuclei segmentation methods in terms of Dice ratio, Dice false positive (Dice FP) and Dice false negative (Dice FN), and Jaccard coefficient. The best performance is labeled in bold.

Method	Dice (%)	Dice FP (%)	Dice FN (%)	Jaccard (%)
ilastik	90.24	8.72	10.80	82.22
U-net	82.50	35.00	0.02	70.21
Attention U-net	83.14	33.70	0.02	77.12
Mask-RCNN	90.22	1.69	17.86	82.19
Cellpose	90.69	2.64	16.14	82.31
RANCA	92.16	9.47	6.21	85.46

shows the statistical mean of these performance measures by six different segmentation approaches performed on 938 nuclei. Our RANCA method shows significant improvement in terms of segmentation accuracy. It is noteworthy that the Dice FP of our method is much lower than other counterpart methods, indicating the accuracy of detected nuclei by our method is very consistent. For the detection recall indicated by Dice FN, our method also shows improved results.

4.2.2. Ablation Study

This section illustrates the advantage of our complete method over the partial method for the densely packed regions. Figure 8a shows the segmentation result if we stop the cascaded models right before the clump identification component. Since there is no fine-tuning step designed for the touching nuclei, the segmentation result is reasonable for single nuclei but does not work well for touching nuclei. Figure 8b shows the segmentation result from cascaded models without the nuclei clump identification scheme. The nuclei clump identification scheme is for regionally adaptive active learning. Without the scheme distinguishing single-nuclei and nuclei-clump regions, touching nuclei are not handled well (e.g., indicated by the orange and blue arrows), despite cascading more models.

We have seen that performance on dense cells depends significantly on the receptive field size, and networks with a matched receptive field will perform better than networks with unmatched ones (either bigger or smaller). We specifically examine segmentation performance with respect to the receptive field size. We evaluate the impact of receptive field size in our proposed RANCA method and the partial method CAC. Specifically, we set the receptive field as a square region and the size of the receptive field to 30, 40, 50, 60, 70, and 80 pixels, respectively. For each setting, we trained the networks on the same training data and then evaluated the nuclear segmentation results in terms of the overlap ratio between the manually labeled nuclei and automatic segmentation results. We also repeated the same procedure to the two networks (RANCA and CAC) for segmenting 837 nuclei, as the Dice ratio shown in

Table 3. The segmentation accuracy with respect to the receptive field size. The ✓ indicates that this operation is used and the best performance is labeled in bold.

Method	Comonents		Size = 30	Size = 40	Size = 50	Size = 60	Size = 70	Size = 80
	CA	RA
CAC	✓		81.65 ± 1.76	83.35 ± 3.26	85.79 ± 4.86	85.98 ± 2.10	88.04 ± 2.44	86.56 ± 3.01
RANCA	✓	✓	82.85 ± 2.44	85.73 ± 1.73	86.14 ± 1.70	86.08 ± 1.77	89.24 ± 1.85	87.92 ± 1.09

. RANCA consistently achieves better segmentation performance as the receptive field size increases from 30 to 80 pixels. We found our RANCA achieved the best performance at 70 pixels for our data. In addition, we found the significant differences are from very dense nuclei clumps and noise areas, such as the regions displayed in the yellow box in Figure 9. The networks with a matched receptive field perform better in close proximity and noise image regions.

For the parameter setting of the nuclei clump detection component, we inspect three selection criteria (ovality, concavity, and size) separately (evaluate one by fixing the other two). By adjusting the parameters (detailed in Section 3.2, the identification result is the union of that of the three criteria), we demonstrate the curve of the Dice ratio with respect to the strictness of each touching nuclei criterion. As shown in Figure 10, by fixing the other two parameters, the segmentation performance is insensitive to the parameter selection of the nuclei clump identification scheme. There are two reasons for this: (1) when the other two parameters are fixed, only a few nuclei are influenced by the variation of one parameter; (2) not only does nuclei clump identification determine the performance, but also the followed specified models play an important role. In addition, in the extreme cases (such as Ovality is at 1, the threshold of ratio of major axis and minor axis of ellipse fitting, all nuclei are identified as clumped nuclei; the cases of Concavity at 0 and Very large size at 0 are similar), the value of Dice decreases sharply. We found the segmentation performance is robust with parameter selection of the nuclei clump identification scheme, and performance deteriorates only in extreme cases.

4.3. Nuclear Counting Application

Recall that we segment each pixel into either nucleus (as shown in Figure 11a), nuclei edge (as shown in Figure 11b), or background after voting across the probability maps. The algorithm can be used to count nuclei in two steps: (1) Nuclei trimming procedure. We first subtract the edge probability from the nuclei probability map. Thus, we can separate the nuclei in the cases where the underlying nuclei are connected. (2) Counting procedure. Counting the number of nuclei eventually turns to the counting of connected regions (eliminating very small regions, for example number of pixels is less than 10) based on the nuclear segmentation results. The result of nuclei counts in nuclei images is shown in Figure 11c, where every connected area is labeled with a unique index. There are in total 256 nuclei detected by our method, which is very close to the ground truth, i.e., 258 nuclei in total.

The quantitative evaluation of nuclei counting is shown in Figure 12. Specifically, the nuclei localization results in four example images are shown in Figure 12a, where we label the center of nuclei with green dots. The white numbers in Figure 12b denote the nuclei index, where the largest index is the total number of nuclei in the cropped region. For inspecting the accuracy of detecting nuclei, the edge (red circle) and ground truth (green dot) are overlaid. Furthermore, we repeat the same nuclei counting method based on the nuclei segmentation results obtained by DeepCell and CAC. Here, we specifically evaluate the advantage of our fine-tuning strategy in separating nuclei clumps by adding CAC (our method without adaptive fine-tuning strategy) for comparison. The total numbers of detected nuclei by these nuclei counting methods are displayed in Figure 12c. As the density of nuclei increases, the counting performances of DeepCell, CAC, and CUBIC become less accurate. It is worth noting that the nuclei counting result here is consistent with the segmentation results shown in the previous section since DeepCell, CAC, and ClearMap have limited capability to separate nuclei in densely packed regions. By contrast, the nuclei counting result using RANCA is much more accurate than other methods, even in the case of high nuclei density.

5. Conclusions

In this paper, we propose an active-learning-based nuclei segmentation method for microscopy images. We formulate nuclei identification to learn a non-linear mapping between local image appearance and the probability of a nucleus at the center of the underlying patch. In order to address the issues of low image contrast and high-intensity inhomogeneity, we developed a cascaded learning network that utilizes both low-level image appearance and high-level contextual features to segment nuclei from microscopy images. The complex learning task is broken down into several intermediate ones in order to train the networks step by step, so our network can progressively improve nuclei segmentation results in a cascaded manner. We further developed an adaptive fine-tuning strategy to actively learn the most challenging problem where multiple nuclei are in close proximity with blurry boundaries. We have evaluated the segmentation results on nuclei images and compared them with other methods. More accurate and robust segmentation results have been achieved, indicating the promising potential of our proposed nuclei segmentation method for use in large microscopy images.

Future work includes (1) improving the segmentation speed by screening the background points, necessitating the application of the cascaded model only on difficult-to-segment image regions; (2) improving the accuracy by extending our current method into a multi-resolution framework; (3) further optimizing the current implementation to fully utilize both CPU and GPU computational resources.