Gain-Assisted Giant Third-Order Nonlinearity of Epsilon-Near-Zero Multilayered Metamaterials

Abstract

We investigate the third-order nonlinear optical properties of epsilon-near-zero (ENZ) Au/dye-doped fused silica multilayered metamaterials in the visible spectral range for TM incident by using nonlocal effective medium theory at different incidence angles. The nonlocal response affects the permittivity of anisotropic metamaterials when the thickness of the layer cannot be much smaller than the incident wavelength. By doping pump dye gain material within the dielectric layer to compensate for the metal loss, the imaginary part of the effective permittivity is reduced to 10⁻⁴, and the optical nonlinear refractive index and nonlinear absorption coefficient are enhanced. The real and imaginary parts of the permittivity are simultaneously minimized when the central emission wavelength of the gain material is close to the ENZ wavelength, and the nonlinear refraction coefficient reaches the order of 10⁻⁵ cm²/W, which is five orders of magnitude larger than that of the nonlinear response of the metamaterial without the gain medium. Our results demonstrate that a smaller imaginary part of the permittivity can be obtained by doping gain materials within the dielectric layer; it offers the promise of designing metamaterials with large nonlinearity at arbitrary wavelengths.

1. Introduction

Recently, epsilon-near-zero (ENZ) materials are attracting great interest due to their optical tunneling effect [1], tailoring of radiation patterns [2], enhanced nonlinearity [3,4,5,6,7,8,9,10,11,12,13,14,15], broadband [16] and high-speed tunable devices [17,18,19,20]. Currently, there are two main mechanisms to realize ENZ with different mechanisms and materials: (1) natural materials near the bulk plasma and phonon resonances; (2) anisotropic metamaterials with alternating stacks of negative permittivity metals, positive permittivity dielectrics [21,22,23,24] and metal pillars embedded in dielectrics [25,26]. The ENZ condition can be achieved at any wavelength of interest by reasonably adjusting the thickness of the metal and dielectric layers and the diameter of the metal pillars in the anisotropic metamaterial. The effective medium theory is valid only when the layer thickness of the anisotropic metamaterial is much smaller than that of the wavelength. However, the layer thickness is too thin; in a few nanometers, it leads to the appearance of unwanted effects in materials, such as: dislocations, island formation and size effects [27]. The effect of spatial dispersion, i.e., nonlocal, becomes increasingly important under the ENZ condition of the anisotropic metamaterial; the appearance of strong nonlocal properties significantly alters the optical properties of the material and can lead to the appearance of additional transverse or longitudinal waves [28,29]. Therefore, it is necessary to design the structure with a more suitable nonlocal effective medium theory [30].

ENZ metamaterials vanish a real part of the permittivity, but metals with a large imaginary part of the permittivity have large optical losses, which seriously affects practical applications and is not conducive to improving the nonlinear response. To circumvent these constraints, optical losses can be compensated for by a gain-doped medium with optically pumped dye molecules [31,32,33,34,35,36] or quantum dot [37,38,39,40,41]. Gain materials have little effect on the real part of the permittivity and greatly reduce the imaginary part of the permittivity at the ENZ wavelength. Due to its negative imaginary part and the essentially invariant real part of the effective permittivity, it improves the Kerr-like optical nonlinear response.

In this paper, the linear and nonlinear optical properties of the ENZ anisotropic multilayered metamaterial with alternating Au/dye-doped fused silica at different incidence angles in the visible range for TM light are investigated by considering spatial dispersion using a more comprehensive theoretical approach, i.e., nonlocal effective medium theory. The gain-doped medium reduces the imaginary part of the transverse permittivity to 10⁻⁴ when the emission wavelength of the gain material is close to the ENZ wavelength. Compared with the metamaterial without gain medium, it has a narrower resonance bandwidth and a 4.92-fold increase in the electric field squared at the ENZ wavelength; the nonlinear coefficient is improved by 5 orders of magnitude.

2. Theory and Method

The multilayered metamaterial is composed of alternating Au/dye-doped fused silica with six cycles arranged on the SiO₂ substrate, as shown in Figure 1. The thickness of gold is 14.9 nm and dye-doped fused silica is 116.2 nm. The optical loss of the metamaterial is compensated for by gain-doped medium with pump dye molecule rhodamine R800. The relative permittivity of this gain-doped medium is a simple Lorentz model [42].

$${\epsilon }_g={\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0$$

(1)

where ${\Gamma }_{pump}$ = $\frac{{\delta }_{abs}{I}_{pump}}{h{f}_{30}}$ = 6.5 × 10⁹ s⁻¹ is the pumping rate, ${\delta }_{abs}$= 3.14 × 10⁻¹⁶ cm² is the absorption cross section, $f_{30}$ = 441 THz is the absorption frequency. ${\epsilon }_0$ is the vacuum permittivity, ${\epsilon }_1$= 2.25 is the dielectric permittivity, ${\delta }_{a }$ is the coupling strength of the polarization density in the emission frequency, ${\lambda }_a=711 \mathrm{nm} , {\omega }_a$= 422 THz is the center emission frequency, $\Delta {\omega }_a$ is the frequency line-width, ${\tau }_{21}$= 500 ps, ${\tau }_{32}={\tau }_{10}=$100 fs, $N_0$ is concentration of the R800 and $N_0$= 1.98 × 10¹⁸ cm⁻³ corresponds to about 3.3 mM. Figure 2 shows the variation in the relative permittivity for the gain-doped medium with respect to the wavelength, the vertical dashed line is the central emission wavelength and the negative imaginary part of the permittivity clearly shows the loss compensated in the gain-doped medium.

The loss is compensated for when the central emission wavelength of the gain medium is close to the ENZ wavelength of the ENZ multilayered metamaterial. The effective medium theory [43,44] is valid only when the layer thickness is small, and the nonlocal effect is not negligible at larger layer thicknesses and under the ENZ condition, which qualitatively changes the optical properties of the structure. The complex effective permittivity in-plane ${\epsilon }_{\parallel }={\epsilon }_{xx}^{nloc} or {\epsilon }_{yy}^{nloc}$ and out-of-plane ${\epsilon }_{\perp }={\epsilon }_{zz}^{nloc}$ is calculated by using the following nonlocal effective medium theory [30].

$${\epsilon }_{xx}^{nloc}=\frac{{\epsilon }_{xx}^{loc}-\frac{\alpha }{12}{k}_0^2{a}^2}{1-\frac{1}{12}{k}_z^2{a}^2}$$

(2)

$${\epsilon }_{yy}^{nloc}={\epsilon }_{yy}^{loc}\left(1+\frac{1}{6}{k}_x^2{a}^2\right)+\frac{a^2}{12k_0^2}(k_z^4-k_x^4)-\frac{\alpha }{12}{k}_0^2{a}^2$$

(3)

$${\epsilon }_{zz}^{nloc}=\frac{{\epsilon }_{zz}^{loc}-\frac{\alpha }{12}{k}_0^2{a}^2}{1+\frac{{\epsilon }_{zz}^{loc}}{{\epsilon }_{xy}^{loc}}\left(\frac{\beta }{12}{k}_x^2{a}^2-\frac{\gamma }{6}{k}_0^2{a}^2\right)}$$

(4)

For the metamaterial without gain medium, where $\mathsf{\alpha }, \mathsf{\beta }, \mathsf{\gamma }$,${\epsilon }_{xx}^{loc}$, ${\epsilon }_{zz}^{loc}$ are:

$$\mathsf{\alpha }=\left[f_m^2{\epsilon }_m+\left(1-f_m^2\right){\epsilon }_d\right]\left[\left(1-f_d^2\right){\epsilon }_m+f_d^2{\epsilon }_d\right]$$

(5)

$$\mathsf{\beta }=\frac{1}{{\mathsf{\epsilon }}_d{\mathsf{\epsilon }}_m}\left[\left(1-2f_m{f}_d\right){\epsilon }_m+2f_m{f}_d{\epsilon }_{\mathrm{d}}\right]\left[2f_m{f}_d{\epsilon }_m+\left(1-2f_m{f}_d\right){\epsilon }_{\mathrm{d}}\right]$$

(6)

$$\mathsf{\gamma }=\frac{1}{{\mathsf{\epsilon }}_d{\mathsf{\epsilon }}_m}\left[f_m^3{f}_2{\mathsf{\epsilon }}_m^3+f_m\left(1-2f_m^2{f}_d+f_d^3\right){\mathsf{\epsilon }}_m^2{\epsilon }_d+f_d\left(1-f_m{f}_d^2+f_m^3\right){\epsilon }_m{\mathsf{\epsilon }}_d^2+f_m{f}_d^3{\mathsf{\epsilon }}_d^3\right]$$

(7)

$${\epsilon }_{xx}^{loc}=f_m{\epsilon }_m+f_d{\epsilon }_d$$

(8)

$${\epsilon }_{zz}^{loc}=\frac{{\epsilon }_d{\epsilon }_m}{f_m{\epsilon }_m+f_d{\epsilon }_d}$$

(9)

For the metamaterial with gain medium, where $\mathsf{\alpha }, \mathsf{\beta }, \mathsf{\gamma }$,${\epsilon }_{xx}^{loc}$, ${\epsilon }_{zz}^{loc}$ are:

$$\begin{gathered} \mathsf{\alpha }=\left[f_m^2{\epsilon }_m+\left(1-f_m^2\right)\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)\right]\left[\left(1-f_d^2\right){\epsilon }_m+f_d^2\left({\epsilon }_1{\epsilon }_0 \\ +\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)\right] \end{gathered}$$

(10)

$$\begin{gathered} \mathsf{\beta }=\frac{1}{{\epsilon }_d{\epsilon }_m}\left[ \left(1-2f_m{f}_d\right){\epsilon }_m+2f_m{f}_d\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)\right] \\ \times \left[2f_m{f}_d{\epsilon }_m+\left(1-2f_m{f}_d\right)\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)\right] \end{gathered}$$

(11)

$$\begin{array}{cc}\mathsf{\gamma }=\frac{1}{{\epsilon }_d{\epsilon }_m}[f_m^3{f}_2{\epsilon }_m^3\hfill & +f_m\left(1-2f_m^2{f}_d+f_d^3\right){\epsilon }_m^2({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\times \frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 )\hfill \\ & +f_d\left(1-f_m{f}_d^2+f_m^3\right){\epsilon }_m{\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\times \frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)}^2\hfill \\ & +f_m{f}_d^3{\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\times \frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)}^3] \hfill \end{array}$$

(12)

$${\epsilon }_{xx}^{loc}=f_m{\epsilon }_m+f_d\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)}\frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)$$

(13)

$${\epsilon }_{zz}^{loc}=\frac{\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)} \frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right){\epsilon }_m}{f_m{\epsilon }_m+f_d\left({\epsilon }_1{\epsilon }_0+\frac{{\delta }_a}{({\omega }^2+i\Delta {\omega }_a\omega -{\omega }_a^2)} \frac{\left({\tau }_{21}-{\tau }_{10}\right){\Gamma }_{pump}}{\left[1+\left({\tau }_{32}+{\tau }_{21}+{\tau }_{10}\right){\Gamma }_{pump}\right]}{N}_0 \right)}$$

(14)

The permittivity of effective medium theory in-plane and out-of-plane is ${\epsilon }_{\parallel }^{loc}={\epsilon }_{xx}^{loc}={\epsilon }_{yy}^{loc}={\epsilon }_{xy}^{loc}$ and ${\epsilon }_{\perp }^{loc}={\epsilon }_{zz}^{loc}$, respectively. $k_0$=$\frac{2\pi }{\lambda }$ is the incident wave vector, $k_x$=$k_0\sin {\theta }_i$, $k_z$=$k_0\cos {\theta }_i, {\theta }_i$ is the incident angle, ${\epsilon }_d$ =$2.25$ is the medium relative permittivity, the metal permittivity ${\epsilon }_m$ can be calculated by the Drude–Lorentz model [45], $f_m=\frac{a_1}{a}$ is the metal filling ratio, $f_d=\frac{a_2}{a}$ is the medium filling ratio, $a_1$ is the metal thickness, $a_2$ is the medium thickness and $a=a_1+a_2$.

The permittivity of TE and TM waves propagating at the angle of θ in the metamaterial is as follows [25]:

$${\epsilon }_{TE}\left(\theta \right)={\epsilon }_{\parallel }$$

(15)

$${\epsilon }_{TM}\left(\theta \right)=\frac{{\epsilon }_{\parallel }{\epsilon }_{\perp }}{{\epsilon }_{\parallel }si{n}^2\left(\theta \right)+{\epsilon }_{\perp }co{s}^2\left(\theta \right)}$$

(16)

It is worth noting that nonlocal parallel permittivities vary from each other in terms of magnitude, as shown in Figure 3a. This means that nonlocality induces an effective biaxiality of the ENZ multilayered metamaterial, even though the structure is spatially uniform in the XY-plane directions. Compared with the local effective medium theory, the ENZ wavelength occurring within the visible spectral range is red-shifted, and the imaginary part is reduced. The ENZ wavelength in the x-direction is 711 nm, and the imaginary part is 0.1207. The nonlocality has a great influence on the out-of-plane permittivity, as shown in Figure 3b. Since we are particularly interested in how the gain medium affects the nonlinearity, only ${\epsilon }_{\parallel }={\epsilon }_{xx}^{nloc}$ is considered later. Figure 4 shows the relative permittivity versus wavelength after adding the gain medium; the gain medium has little effect on the real part of the permittivity and a great effect on the imaginary part, and the imaginary part of the permittivity is reduced to 5.36 × 10⁻⁴ at the ENZ wavelength 711 nm.

3. Results and Discussions

3.1. Linear Response

The optical properties are affected not only by changing the layer thickness and permittivity, but also by different incidence angles. The characteristics of reflectance R, transmittance T and absorption A of the metamaterial without gain medium for TM wave at different incidence angles are shown in Figure 5, absorption A = 1 – T − R, and the dashed line is the ENZ wavelength 711 nm. The plasmon–polaritonic coupling of the metal–dielectric interface greatly confines the electric field intensity in the dielectric layer [46,47,48,49], leading to multiple distinct resonances of reflectance, transmittance and absorption between wavelengths of 450–800 nm. The metal–dielectric–metal sandwich structure in the multilayered metamaterial is similar to an F-P resonant cavity, which enhances the electric field intensity and, therefore, produces transmittance windows, absorption peaks and reflectance dips near the ENZ wavelength. The resonances of transmittance, reflectance and absorption exhibit a similar blueshift phenomenon with increasing incident angle, and the transmittance of the multilayered metamaterial increases with increasing incident angle, but the reflectance and absorption conversely decrease.

In Figure 6, we reveal transmittance characteristics of the structure with gain medium at different incident angles. The transmittance resonance peak at 711 nm in Figure 6 is narrower than that in Figure 5a because of the reduced permittivity after adding the gain medium. The structure also has absorption at 711 nm, but the absorption is smaller than the gain, so there is a narrow-band amplification of the transmittance peak, and transmittance resonance peaks at the remaining wavelengths are similar to Figure 5a, because the permittivity is the same as that of the structure without gain medium. According to the continuity of the electric displacement field for the boundary conditions at the interface for ENZ metamaterials, that is, $\left|E\right|\propto {\epsilon }^{-1}\left|E_0\right|$, ENZ metamaterials with the gain medium have a strong squared of electric field $({\left|E\right|}^2$) enhancement due to the small magnitude of the permittivity ($\epsilon$), which is 4.92 times larger than in the case without the gain medium at normal incidence. The electric field intensity at the ENZ wavelength also decreases with increasing angle due to the blueshift of the transmittance resonance wavelength and the decrease in the transmittance peak, as shown in Figure 7d–f.

3.2. Nonlinear Response

The nonlinear refractive index and nonlinear absorption coefficient of the ENZ multilayered metamaterial were investigated using nonlinear effective medium theory [46]. Here, the effective nonlinear third-order susceptibility ${\chi }_{eff}^3$ of the structure is a weighted average of the constituent materials; since the nonlinear susceptibility of dielectric ${\chi }_d^{\left(3\right)}$ is much smaller than that of the Au ${\chi }_m^{\left(3\right)}$ and can be neglected, only the ${\chi }_m^{\left(3\right)}$ can be considered to the ${\chi }_{eff}^3$ in the ENZ multilayered metamaterial. Au in different material configurations, growth conditions and pulse duration will have different values for the third-order susceptibility, ranging from 10⁻¹⁴ to 10⁻¹⁹ m²V⁻² [50,51,52,53]. However, we are interested in the effect of the gain medium on the third-order nonlinearity rather than optimizing the optimal third-order nonlinearity. In what follows, we assume that Au is dispersion-less [23] and that the third-order susceptibility of Au is ${\chi }_{xx}^{\left(3\right)}={\chi }_{yy}^{\left(3\right)}={\chi }_{zz}^{\left(3\right)}=$ 10⁻¹⁷ m²V⁻² [54]. The third-order nonlinear susceptibility of the ENZ multilayered metamaterial parallel to the interface and perpendicular to the interface is as follows [49].

$${\chi }_{\parallel }^{\left(3\right)}=f_m{\chi }_m^{\left(3\right)}$$

(17)

$${\chi }_{\perp }^{\left(3\right)}=f_m{\chi }_m^{\left(3\right)}{\left|\frac{{\epsilon }_{\perp }^{nloc}}{{\epsilon }_m}\right|}^2{\left(\frac{{\epsilon }_{\perp }^{nloc}}{{\epsilon }_m}\right)}^2$$

(18)

where $\frac{{\epsilon }_{\perp }^{nloc }}{{\epsilon }_m}$ displays the local field enhancement factor in the metallic component.

For the TM wave at all incidence angles, the third-order susceptibility is derived as follows:

$${\epsilon }_{TM}^{NL}\left(\theta \right) =\frac{{\epsilon }_{\parallel }^{NL}{\epsilon }_{\perp }^{NL}}{{\epsilon }_{\parallel }^{NL}si{n}^2\left(\theta \right)+{\epsilon }_{\perp }^{NL}co{s}^2\left(\theta \right)}$$

(19)

where ${\epsilon }_{\parallel }^{NL}$ and ${\epsilon }_{\perp }^{NL}$ are the in-plane and out-of-plane nonlinear permittivity, respectively [23]:

$${\epsilon }_{\parallel }^{NL} ={\epsilon }_{\parallel }+3{\chi }_{\parallel }^{\left(3\right)}{\sin }^2\left(\theta \right){\left|E\right|}^2$$

(20)

$${\epsilon }_{\perp }^{NL} ={\epsilon }_{\perp }+3{\chi }_{\perp }^{\left(3\right)}{\sin }^2\left(\theta \right){\left|E\right|}^2$$

(21)

By replacing ${\epsilon }_{\parallel }^{NL}$ and ${\epsilon }_{\perp }^{NL}$ in Equation (19) with Equations (20–21), the nonlinearity in the denominator is too small compared with linearity, so the effects of the nonlinearity in the denominator and the electric field ${\left|E\right|}^4$ can be neglected.

$${\epsilon }_{TM}^{NL}\left(\theta \right)\approx {\epsilon }_{TM}\left(\theta \right)+3{\epsilon }_{TM}\left(\theta \right){\sin }^2\left(\theta \right)\left(\frac{{\chi }_{\parallel }^{\left(3\right)}}{{\epsilon }_{\parallel }^{nloc}}+\frac{{\chi }_{\perp }^{\left(3\right)}}{{\epsilon }_{\perp }^{nloc}}\right){\left|E\right|}^2$$

(22)

To make a further approximation, assuming that the nonlinear component is much smaller than the linear component, Snell’s law can be written as ${\epsilon }_{TM}^{NL}\left(\theta \right){\sin }^2\left(\theta \right)$=${\epsilon }_{TM}\left(\theta \right){\sin }^2\left(\theta \right)$=${\epsilon }_i{\sin }^2\left({\theta }_i\right)$. ${\theta }_i$ is the angle of incidence and ${\epsilon }_i$ is the permittivity of the air medium, therefore, Equation (22) can be rewritten as:

$${\epsilon }_{TM}^{NL}\left({\theta }_i\right)\approx {\epsilon }_{TM}\left({\theta }_i\right)+3{\epsilon }_i{\sin }^2\left({\theta }_i\right)\left(\frac{{\chi }_{\parallel }^{\left(3\right)}}{{\epsilon }_{\parallel }^{nloc}}+\frac{{\chi }_{\perp }^{\left(3\right)}}{{\epsilon }_{\perp }^{nloc}}\right){\left|E\right|}^2$$

(23)

$${\chi }_{TM}^{\left(3\right)}\left({\theta }_i\right)\approx {\epsilon }_i\left(\frac{{\chi }_{\parallel }^{\left(3\right)}}{{\epsilon }_{\parallel }^{nloc}}+\frac{{\chi }_{\perp }^{\left(3\right)}}{{\epsilon }_{\perp }^{nloc}}\right)$$

(24)

The complex nonlinear response is:

$${\tilde{n}}_2=\frac{3}{4c{\epsilon }_0\mathrm{Re}\left(n_0\right)n_0}{\chi }_{eff}^3$$

(25)

where ${\epsilon }_0$ is the permittivity in a vacuum, $n_0$ is the complex effective linear refractive index, $\mathrm{Re}\left(n_0\right)$ is the real part of the complex effective linear refractive index and $c$ is the speed of light. The nonlinear refraction coefficient and nonlinear absorption coefficient are as follows:

$$n_2=Re\left({\tilde{n}}_2\right)$$

(26)

$$\mathsf{\beta }=\frac{4\mathsf{\pi }}{\mathsf{\lambda }}\mathrm{Im}\left({\tilde{n}}_2\right)$$

(27)

We investigated the intensity-dependent refraction index n of the multilayered metamaterial. The nonlinear refractive index change is greater than the linear refractive index change under ENZ conditions, which violates the nonlinear perturbation theory [3]. Therefore, the conventional intensity-dependent refractive index, $n\left(I\right)=n_0+n_{2}I$, is not applicable; we directly use the susceptibility to calculate the intensity-dependent refractive index, i.e., $n\left(I\right)=\sqrt{{\epsilon }^{\left(1\right)}+3{\chi }^{\left(3\right)}{\left|E\right|}^2}$. The incident light intensity I directly affects the permittivity of the gain medium as in Equation (1). The permittivity imaginary $Im({\epsilon }_{\parallel })$ is closest to 0 as the incident light intensity increases to 6.15 MW/cm², and the refractive index Re(n) = 5.57 is at its maximum at I = 6.15 MW/cm², as shown in Figure 8b, but the nonlinear bandwidth becomes narrower than I = 6.05 MW/cm². When I > 6.15 MW/cm², Re(n) decreases due to the increase in $Im({\epsilon }_{\parallel })$. We use I = 6.05 MW/cm² in the following after considering the nonlinear coefficient and the bandwidth size. For metamaterials without the gain medium, Re(n) in Figure 8a is much smaller than that in Figure 8b with the gain medium. The above results show that multilayered metamaterials with gain media exhibit an order of magnitude of nonlinear refractive index change under ENZ conditions.

The metamaterial displays a strong wavelength-dependent nonlinear response; the conversion between self-focusing to self-defocusing and saturated absorption and anti-saturated absorption can be achieved by controlling the wavelength. For the metamaterial without gain medium, the $\left|n_2\right|$ is the maximum and reaches the order of 10⁻¹⁰ cm²/W at the wavelength of 717.8 nm, as shown in Figure 9a. For the metamaterial with gain medium, as shown in Figure 10a, the complex linear refractive index is well reduced and the third-order nonlinear susceptibility is increased due to the reduction in the imaginary part of the dielectric constant at 710.8 nm, thus, the $\left|n_2\right|$ is the maximum and reaches the order of 10⁻⁵ cm²/W, which is an increase of 5 orders of magnitude compared with the nonlinear refractive index for normal incidence in Figure 9a. Similarly, for the metamaterial without gain medium, the $\left|\mathsf{\beta }\right|$ at 707.3 nm is the maximum and reaches the order of 10⁻⁵ cm/W, as shown in Figure 9b. For the metamaterial with gain medium, as shown in Figure 10b, the $\left|\mathsf{\beta }\right|$ is the maximum at the wavelength of 710.8 nm and reaches the order of 1 cm/W, which is an increase of 5 orders of magnitude in the nonlinear absorption coefficient compared with Figure 9b.

Table 1. Comparisons of the third-order nonlinear optical coefficients of different references.

References	Structures	$\mathbf{Wavelength} \left(\mathbf{nm}\right)/{\mathit{n}}_2({\mathbf{cm}}^2/\mathbf{W})$	$\mathbf{Wavelength} \left(\mathbf{nm}\right)/\mathsf{\beta } (\mathbf{cm}/\mathbf{W})$
Our work	Multilayered metamaterial with a gain medium	710.8/10−5	710.8/100
[3]	ITO film	1240/10−10	1260/10−6
[4]	AZO film	1310/10−13	1340/10−8
[5]	Metasurface	1240/10−9	1260/10−5
[10]	Metasurface	1250/10−9	1300/10−4
[23]	Multilayered metamaterial	470/10−8	470/10−3
[24]	Multilayered metamaterial	500/10−9	500/10−4

shows the comparisons of nonlinear coefficients (nonlinear refraction coefficient and nonlinear absorption coefficient) for similar systems, and the results of this article are shown in bold. The nonlinear refraction coefficient and nonlinear absorption coefficient reach the order of 10⁻⁵ cm²/W and 1 cm/W, respectively, which are several orders of magnitude larger than the results from other literature. Therefore, the gain-doped medium improves the nonlinear response, and the dominant mechanism for improving the nonlinearity is the factor $1/\mathrm{Re}\left(n_0\right)n_0$.

.

Finally, we compare nonlinear coefficients of gain media with different incidence angles and different concentrations: 0 mM in Figure 11, 2.5 mM in Figure 12, 3.3 mM in Figure 13. Since the imaginary part of the permittivity increases and the electric field intensity coupled in the medium decreases with the increasing angle, the absolute value of the nonlinear coefficient decreases over the entire wavelength range. We consider concentrations of R800 dye molecules in the dielectric layer from 0 mM to 3.3 mM; on the one hand, the reduction in the imaginary part of the effective permittivity in the plane, $Im\left({\epsilon }_{\parallel }\right)$≈ 0.1207 for 0 mM to $Im\left({\epsilon }_{\parallel }\right)$ ≈ 10⁻⁴ for 3.3 mM, i.e., the loss is well compensated. On the other hand the narrower resonance bandwidth leads to an increase in the intensity of the coupled electric field in the dielectric layer with increasing concentration of the gain medium. The nonlinear coefficients, near the ENZ wavelength, are increased by 1 order and 5 orders of magnitude for concentrations of 2.5 mM and 3.3 mM, respectively. Further, a broadband nonlinear response is shown for 0 mM concentration. When the gain medium concentration increases and loss is compensated, the nonlinear displays an angle/spectrum narrowband characteristic. Therefore, the nonlinearity of the ENZ multilayered metamaterial must be improved with an appropriate concentration of the gain medium.

Table 2. Comparisons of the third-order nonlinear optical coefficients of different doping concentrations.

Concentrations	${\mathit{n}}_2({\mathbf{cm}}^2/\mathbf{W})$	$\mathsf{\beta } (\mathbf{cm}/\mathbf{W})$
3.3 mM	10−5	100
2.5 mM	10−9	10−4
0 mM	10−10	10−5

shows the comparisons of nonlinear coefficients (nonlinear refraction coefficient, nonlinear absorption coefficient) at different doping concentrations.

4. Conclusions

In conclusion, the linear and nonlinear properties of the ENZ multilayered metamaterial are investigated based on the nonlocal effective medium theory, and the ENZ wavelength of interest can be adjusted arbitrarily by adjusting the thickness of the dielectric and metal layers. The metal–dielectric–metal in the ENZ multilayered metamaterial is similar to the F-P cavity; the electric field intensity can be coupled in the dielectric layer, but metals have large optical losses, leading to a large imaginary part of the permittivity, which seriously affects practical applications and is not conducive to improving the nonlinear response. Therefore, the doping of R800 dye molecules with a concentration of 3.3 mM in the dielectric layer can be used to reduce the imaginary part of the permittivity in the ENZ multilayered metamaterial to 5.36 × 10⁻⁴. Compared with the metamaterial without gain medium, the squared electric field is enhanced by 4.92 times, and the nonlinear coefficient is increased by 5 orders of magnitude. Moreover, the conversion of self-focusing and self-defocusing as well as saturable absorption and anti-saturable absorption can be achieved by controlling the wavelength, and the absolute value of the nonlinear coefficient decreases with increasing angle. We believe that our investigation of the gain-doped ENZ multilayered metamaterial can lead to important applications in nonlinear optics and quantum optics.