*2.2. Eigenvector Processing*

Color vision deficiency cannot be understood easily by most people with normal vision. Thus, it is necessary to use simulation tools to create synthetic images for ordinary viewers to understand what are seen by the colorblind people [35]. Some well-known tools include Colblindor [6] and LMS [25], and there are also several websites to perform the simulation online (For example, Coblis Color Blindness Simulator (http://www.color-blindness.com/coblis-color-blindness-simulator), Color Blindness Simulator (http://www.etre.com/tools/colourblindsimulator), and Vision Simulator (http: //www.webexhibits.org/causesofcolor/2.html)). In this work, Machado's approach is adopted for our color vision deficiency simulation [35]. It utilizes a physiology based simulation model to achieve the sensation of cones in human visual perception. The simulation is to shift the pigments of the responding curve of spectral sensitivity functions as shown in Figure 3. Anomalous trichromacy can be simulated by shifting the sensitivity of the L, M, and S cones in the following ways:


The elements of the transformation matrix Γ can be derived by

$$f(\lambda, R, G, B)\_{WS, YB, G} = \rho\_{R, G, B} \int \Phi\_{R, G, B}(\lambda) f(\lambda)\_{WS, YB, G} \, d\lambda \tag{5}$$

where *φR*,*G*,*<sup>B</sup>* is the spectral power distribution function, and *ρR*,*G*,*<sup>B</sup>* is a normalization factor. Thus, Γ is the projection of the spectral power distributions of RGB primaries onto a set of basic functions *f*(*λ*, *R*, *G*, *B*)*WS*,*YB*,*RG*. That is,

$$
\Gamma = \begin{bmatrix}
f(R)\_{WS} & f(G)\_{WS} & f(B)\_{WS} \\
f(R)\_{YB} & f(G)\_{YB} & f(B)\_{YB} \\
f(R)\_{RG} & f(G)\_{RG} & f(B)\_{RG} \end{bmatrix} . \tag{6}
$$

**Figure 3.** The cone spectral sensitivity functions at all wavelengths in the visible range. (**a**) Responding curve [36]. (**b**) Spectral response functions for the opponent channels [34].

This model is based on the stage theory of human color vision, and is derived from the data reported in electro-physiological study [34]. Let Φ*CVD* be the matrix that maps RGB to the opponent-color space of normal trichromacy, then the simulation of dichromatic vision is obtained by the transformation

$$
\begin{bmatrix} R\_s \\ G\_s \\ B\_s \end{bmatrix} = \Phi\_{CVD} \begin{bmatrix} R \\ G \\ B \end{bmatrix} . \tag{7}
$$

By definition, an eigenvector is the non-zero vector mapped by a given linear transformation of a vector space onto a vector that is the product of the original vector multiplied by a scalar. Thus, the algorithm counts the eigenvectors of the covariance matrix from the images in *Y-B*, *R-G* of the *λ*, *Y-B*, *R-G* opponent color space, i.e.,

$$\dot{\varepsilon}[v,d] = \dot{\varepsilon}ig(\varepsilon ov(I\_{Y-B}, I\_{R-G})) \tag{8}$$

where *eig* is the function of eigenvalue and eigenvector, and *IY*−*<sup>B</sup>* and *IR*−*<sup>G</sup>* are the *Y-B* and *R-G* images, respectively. On the left hand side of the equation, *d* is the generalized eigenvalue, and *v* is a 2 × 2 matrix since the covariance *cov* is a 2 × 2 matrix derived from a pair of *n* × 1 images given by the covariance matrix

$$cov(X, Y) = \frac{\sum\_{i=1}^{n} (X\_i - \bar{X})(Y\_i - \bar{Y})}{(n - 1)} \tag{9}$$

For the original and CVD simulation images shown in Figure 4, the characteristics of the associated eigenvectors are illustrated in Figure 5. The black line (at about 91◦) indicates the eigenvector of the original image. For protanopia (red line about 150◦) and deuteranopia (green line about 140◦), the eigenvectors lead the one associated with the original image. The eigenvector of tritanopia (blue line at about 80◦) is behind the original image case. Our objective is to recover the angle difference

between the normal and color vision deficiency images, and use it to re-color the image. The difference image when observed by normal viewers and the color vision deficiency simulation is defined by

$$I\_{diff} = \sqrt{(I\_n(\Upsilon B) - I\_c(\Upsilon B))^2 + (I\_n(RG) - I\_c(RG))^2} \tag{10}$$

where *In* and *Ic* represent the intensity observed by a normal viewer and obtained from the color vision deficiency simulation, respectively.

(**a**) Original (**b**) Protanopia

(**c**) Deuteranopia (**d**) Tritanopia

**Figure 4.** The three types of color vision deficiency simulation using Machado's approach [35] with sensitive 0.6 and the matrix Φ*CVD* as shown in Table 3.



**Figure 5.** Eigenvectors of the covariance matrix of the images shown in Figure 4.

An example of protanopia simulation is illustrated in Figure 6. The difference image is shown in Figure 6c, and a binary image for better illustration is shown in Figure 6d. Our objective is to recover the angle and difference between the normal and CVD images and use the information to re-color the CVD simulation images.

**Figure 6.** An example of Protanopia simulation. (**a**) is the original image and (**b**) is the Protanopia simulation result. (**c**) is the difference of (**a**,**b**) computed in the *λ*, *Y-B*, *R-G* color space. (**d**) is the binarized version of (**c**) for better illustration.

#### *2.3. Color Warping*

The color values of the images are transformed from RGB to the opponent color space *λ*, *Y-B*, *R-G* using Equations (3) and (4). It is assumed that color vision deficiency does not affected by the brightness, so the value *λ* corresponding to the luminance is keep intact. To define the warping range with the angle of an eigenvector, we construct twelve pure colors in RGB using the values of 0, 150 and 255. Figure 7 shows the range of missing chroma of color vision deficiency. The area within the two green lines is the red chroma missing for protanopia and deuteranopia, and the area within th two purple lines is the blue chroma missing for tritanopia. The color points represented in the Cartesian coordinates are then transformed to the polar coordinates by

$$\theta = \tan^{-1} \frac{\underline{y}}{\underline{x}} \tag{11}$$

for processing. The angle *θ* associated with the eigenvector in the *λ*, *Y-B*, *R-G* color space is used to derive the range to be processed. Since the image is now in the opponent color space, the range is defined by the angle of the simulation vector to the opposite angle of the simulation vector. Finally, the warping range is defined by the vertical angle of the original vector to the opposite angle of the simulation vector. An example is illustrated in Figure 8, the green area is warped to the red area for image re-coloring.

**Figure 7.** The pure colors RGB (0, 150, and 255).

**Figure 8.** An illustration of the color warping range from the green area to the red area.

The new color angle is derived from the original color angle by

$$
\theta\_{ncv} = \frac{\theta\_\perp - \theta\_{op}}{\pi} \cdot (\theta - \theta\_{op}) \tag{12}
$$

where the angles of color points are defined in the range of [−*π*, *π*], *θ*<sup>⊥</sup> is the angle of vector orthogonal to the original vector, and *θop* is the angle of vector opposite to the color vision deficiency simulation vector.

When the image is converted from RGB to the *λ*, *Y-B*, *R-G* color space, it is in a limited range of color space representation. We need a constraint to avoid the luminance overflow problem, which will make colors not smooth after converted back to the RGB color representation. In our approach, a convex hull is adopted for the color constraint due to its simplicity for boundary derivation. Figure 9a–d illustrate the full-color images constructed using 256<sup>3</sup> pixels, i.e., the resolution of 4096 × 4096, and the corresponding convex hull is shown in Figure 9e (the red lines). The formula used for conversion is given by

$$
\rho\_{nvw} = \rho \times \frac{\rho(\theta\_{nvw})}{\rho(\theta)}\tag{13}
$$

where *ρ* is the original value, *ρ*(*θnew*) is the value of the convex hull at *θnew*, and *ρ*(*θ*) is the value of the convex hull at *θ*. The resulting image in the *λ*, *Y-B*, *R-G* color space is then transformed back to the RGB color space for display.

**Figure 9.** (**a**) Three types of color vision deficiency simulation using a full-color image with 2563 pixels. The image resolution is 4096 × 4096. (**e**) The convex hulls of the images in (a–d). All types of CVD simulation cover only a part of the convex hull of the original full color imag.
