Existence Results for Neumann Problem with Critical Sobolev–Hardy Exponent and Choquard-Type Nonlinearity

Abstract

We consider a Neumann problem for the fractional Laplacian involving a nonlocal Choquard-type nonlinearity and Sobolev–Hardy exponent. Under suitable assumptions on the data and using the Nehari manifold method, we discuss the existence problem in several subcritical and critical cases.

1. Introduction and Results

Let $\mathsf{\Omega }\subset {\mathbb{R}}^N$ be a smooth bounded domain with smooth boundary $\partial \mathsf{\Omega }$. In this paper, we study the following parametric Sobolev–Hardy problem:

$$\left\{\begin{array}{c}{\displaystyle {(-▵)}^{\alpha }\phi -\kappa \frac{\phi }{{\left|x\right|}^{2\alpha }}=h\left(x\right){\int }_{\mathsf{\Omega }}\frac{{\left|\phi \right|}^p}{{|x-y|}^{\mu }}dy{\left|\phi \right|}^{p-2}\phi ,x\in \mathsf{\Omega },}\hfill \\ {\displaystyle \frac{\partial \phi }{\partial n}=\lambda f\left(x\right)\frac{{\left|\phi \right|}^{q-2}\phi }{{\left|x\right|}^{\beta }},x\in \partial \mathsf{\Omega },\lambda >0.}\hfill \end{array}\right.$$

(1)

Given $\alpha \in (0,1)$, in this problem, ${(-▵)}^{\alpha }\phi$ is the fractional Laplacian operator, also known as the Riesz fractional derivative, defined by

$${(-\mathsf{\Delta })}^{\alpha }\phi \left(x\right)=C(N,\alpha )\mathrm{p}.\mathrm{v}.{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{\phi \left(x\right)-\phi \left(y\right)}{{|x-y|}^{N+2\alpha }}}\mathrm{d}y,$$

where $\mathrm{p}.\mathrm{v}.$ is the Cauchy principal value of the integral, and $C(N,\alpha )$ is the normalization constant given as

$$C(N,\alpha )=2^{2\alpha -1}{\pi }^{-{\displaystyle \frac{N}{2}}}{\displaystyle \frac{\mathsf{\Gamma }\left(\frac{N+2\alpha }{2}\right)}{\left|\mathsf{\Gamma }\right(-\alpha \left)\right|}},$$

where $\mathsf{\Gamma }$ denotes the Gamma function, see, for example, Frank–Lieb–Seiringer ([1], Lemma 3.1). The constant exponents involved in (1) have to satisfy the following requirements: $0\le \beta \le 2\alpha <N$, $0<\mu <N$, $1<2p<2$, $1<q\le 2_{\beta ,\alpha }^{*}=2(N-\beta )/(N-2\alpha )$, and $2<2_{\beta ,\alpha }^{*}<2_{\alpha }^{*}=2N/(N-2\alpha )$, where $2_{\beta ,\alpha }^{*}$ is the fractional critical Hardy–Sobolev exponent. If ${\kappa }_0$ denotes the best Hardy constant in the fractional Hardy inequality (see Section 2, and the discussion in [1] too), we assume

$${\displaystyle 0<\kappa <{\kappa }_0=4^{\alpha }\frac{{\mathsf{\Gamma }}^2\left(\frac{n+2\alpha }{4}\right)}{{\mathsf{\Gamma }}^2\left(\frac{n-2\alpha }{4}\right)}.}$$

Further, $h\left(x\right)\in L^{\infty }\left(\mathsf{\Omega }\right)$ and $f\left(x\right)\in L^{\infty }(\partial \mathsf{\Omega })$ are the reaction coefficient and the boundary coefficient, respectively. In the boundary condition, $\partial /\partial n$ denotes the generalized directional derivative (conormal derivative) of $\phi$, with $n(·)$ being the outward unit normal on $\partial \mathsf{\Omega }$. Such a directional derivative is dictated by the nonlinear Green’s identity (see, for example, Gasiński and Papageorgiou [2]).

If $r\in (1,+\infty )$, the study of elliptic r-Laplacian problems (driven by the operator ${▵}_r\phi ={div\left(\right|\nabla u|}^{r-2}\nabla u)$ for all $u\in W_0^{1,r}\left(\mathsf{\Omega }\right)$) with competing nonlinearities under different boundary conditions has been largely refined in recent decades, for example, we mention the works by Cherfils and Il’yasov [3] (for the sum of two r-Laplacian operators with different exponents) and Papageorgiou et al. [4] (for the double-phase operator). Such problems are considered useful models in the analysis of electrorheological fluids, in image processing, and in the context of nonlinear elasticity theory; hence, the reader can refer to Acerbi and Mingione [5] and Afrouzi and Ghorbani [6] and the references therein. In the case of a single Laplacian operator ($r=2$), Chen [7] focused on the following Dirichlet problem

$$\left\{\begin{array}{c}-▵\phi -{\displaystyle \frac{\mu }{{\left|x\right|}^2}}\phi ={\left|\phi \right|}^{2^{*}-2}\phi +\lambda \phi ,x\in \mathsf{\Omega },\hfill \\ {\displaystyle \phi \left(x\right)=0,x\in \partial \mathsf{\Omega },}\hfill \end{array}\right.$$

(2)

where $2^{*}={\displaystyle \frac{2N}{N-2}}$, $0\le \mu <\overline{\mu }={\left({\displaystyle \frac{N-2}{2}}\right)}^2$, and $\lambda >0$ is a parameter. The main source of difficulty here is in the lack of compactness for the Palais–Smale sequences ($\left(PS\right)$-sequences for short) of the functional associated with (2). This way, variational methods cannot be applied directly; hence, the approach to the existence problem in [7] is based on the Linking theorem and delicate energy estimates for the functional. The similar problem is investigated by Cao and Peng [8] to conclude the existence of sign-changing solutions by using Ljusternik–Schnirelman theory (see Zeidler [9]) and an (subcritical) approximating problem to (2).

Given $\alpha \in (0,1)$, Bhakta et al. [10] considered the following fractional Hardy–Sobolev equation

$${\displaystyle {(-▵)}^{\alpha }\phi -\gamma \frac{\phi }{{\left|x\right|}^{2\alpha }}=K\left(x\right)\frac{{\left|\phi \right|}^{2_{\alpha }^{*}\left(t\right)-2}\phi }{{\left|x\right|}^t}+f\left(x\right),x\in {\mathbb{R}}^N,}$$

(3)

where $N>2\alpha$, $0\le t<2\alpha <N$, $2_{\alpha }^{*}\left(t\right)={\displaystyle \frac{2(N-t)}{N-2\alpha }}$, $0<\gamma <{\gamma }_{N,\alpha }$, here ${\gamma }_{N,\alpha }$ is the best Hardy constant in the fractional Hardy inequality. Hence, they obtained the existence and multiplicity results for constant sign solutions (more precisely, positive solutions). The approach is based on the classification of certain $\left(PS\right)$-sequences for the functional associated to (3), performing a profile decomposition of the $\left(PS\right)$-sequence in general Hilbert spaces (to overcome the already mentioned lack of compactness). Further, under a Neumann nonlocal boundary condition, Irzi and Kefi [11] studied the existence of solution to a fractional $\overline{r}$-Laplacian problem ($\overline{r}=r(·,·)$ is a suitable continuous function defined on a smooth bounded domain of ${\mathbb{R}}^N$). Regarding the study of fractional operator theory, Muslih et al. [12,13] effectively resolved linear and specific nonlinear problems in fractional-dimensional spaces through Fourier transform methods, while Lima et al. [14] established a geometric interpretation of the relationship between critical exponents and fractal dimensions. Differently from the previous works, this time, the approach is based on the Ekeland principle, together with variational tools. Another interesting contribution is due to Fan [15], who showed the existence of nontrivial weak solutions to fractional Choquard problems of the form

$$\left\{\begin{array}{c}{\displaystyle {(-▵)}^{\alpha }\phi ={f\left(x\right)\left|\phi \right|}^{q-2}\phi +{\int }_{\mathsf{\Omega }}\frac{{\left|\phi \right|}^p}{{|x-y|}^{\mu }}dy{\left|\phi \right|}^{p-2}\phi ,x\in \mathsf{\Omega },}\hfill \\ {\displaystyle \phi =0,x\in {\mathbb{R}}^N\backslash \mathsf{\Omega },}\hfill \end{array}\right.$$

(4)

where $0<\alpha <1$, $(2N-\mu )/N\le p\le 2_{\mu ,s}^{*}/(N-2\alpha )$, and $0<\mu <N$. By using variational tools and the Nehari manifold method, the author investigated both subcritical and critical nonlinearities, and obtained the results stated in Theorems 1.2 and 1.3, respectively. Fan [16] also established similar results in the case of a fractional Choquard equation with Kirchhoff weight, to underline the effectiveness of the strategy in dealing with various classes of differential problems. Turning to the non-fractional setting, some contributions in this direction are the works by Brown and Zhang [17], where both existence and non-existence results for positive solutions to a semilinear elliptic boundary value problem with a sign changing weight function are discussed, de Albuquerque and Silva [18], where the Nehari manifold method is applied to a class of Schrödinger equations with indefinite weight functions, and Gasiński and Winkert [19], where a double phase problem is investigated to obtain the existence and multiplicity results.

Inspired by these works, the purpose of our paper is in discussing the existence and multiplicity of weak solutions, with sign information, to problem (1) by using the Nehari manifold method. Differently from the previous works (recall (2)–(4)), the features of our problem are the presence of the Hardy term, together with a Choquard-type nonlinearity and a Neumann boundary condition, which makes the proof that the energy functional satisfies the correlation property and the associated parameter settings difficult. Further, we distinguish the subcritical case ($q<2_{\beta ,\alpha }^{*}$) and the critical case ($q=2_{\beta ,\alpha }^{*}$). Specifically, the critical case presents greater challenges, as it necessitates information regarding the asymptotic behavior of solutions to the limiting problem at both zero and infinity. More precisely, we establish the following theorems.

Theorem 1.

If $0<\alpha <1$, $0<\mu <N$, $0\le \beta \le 2\alpha <N$, $0<\kappa <{\kappa }_0$ and $1<2p<2<q<2_{\beta ,\alpha }^{*}<2_{\alpha }^{*}$, then there exists $\overline{\lambda }>0$ such that problem (1) has at least two positive solutions for all $\lambda \in (0,\overline{\lambda })$.

Theorem 2.

If $0<\alpha <1$, $0<\mu <N$, and $0\le \beta \le 2\alpha <N$, $0<\kappa <{\kappa }_0$ and $1<2p<2<q=2_{\beta ,\alpha }^{*}<2_{\alpha }^{*}$, then there exists ${\overline{\lambda }}^{*}>0$ such that problem (1) has at least two positive solutions for all $\lambda \in (0,{\overline{\lambda }}^{*})$.

The rest of the paper is organized as follows. In Section 2, we collect the mathematical background. In Section 3, we discuss the Nehari manifold for the energy functional associated with problem (1). In Section 4 and Section 5, we present the proofs of Theorem 1 and Theorem 2, respectively.

2. Functional Setting

The analysis of problem (1) requires the use of fractional Sobolev spaces. A comprehensive presentation of such spaces can be found in the monographies of Di Nezza et al. [20] and Molica Bisci et al. [21]. Given $\alpha \in (0,1)$, we define the fractional Sobolev space $W^{\alpha ,2}\left(\mathsf{\Omega }\right)=H^{\alpha }\left(\mathsf{\Omega }\right)$ as follows

$$H^{\alpha }\left(\mathsf{\Omega }\right)=\left\{\phi \in L^2\left(\mathsf{\Omega }\right):{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}}\mathrm{d}x\mathrm{d}y<+\infty \right\}.$$

This vector space is equipped with the norm given by

$${\parallel \phi \parallel }_{H^{\alpha }\left(\mathsf{\Omega }\right)}={\parallel \phi \parallel }_{L^2\left(\mathsf{\Omega }\right)}+{\left({\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}}\mathrm{d}x\mathrm{d}y\right)}^{{\displaystyle \frac{1}{2}}},$$

where the first term is the usual norm for the space $L^r\left(\mathsf{\Omega }\right)$ (here $r=2$), that is ${\parallel \phi \parallel }_r={\left({\int }_{\mathsf{\Omega }}{\left|\phi \right|}^r\mathrm{d}x\right)}^{{\displaystyle \frac{1}{r}}}$, and the second term is the so-called “Gagliardo (semi)norm” of $\phi$. Then, $H^{\alpha }\left(\mathsf{\Omega }\right)$ becomes a Banach space.

For $\alpha \in (0,1)$, the fractional Sobolev space $H^{\alpha }$ can also be defined as the completion of $C_c^{\infty }\left({\mathbb{R}}^N\right)$ under the norm

$$\begin{array}{cc}\hfill {\parallel \phi \parallel }_{H^{\alpha }\left(\mathsf{\Omega }\right)}^2& ={\int }_{\mathsf{\Omega }}{\left|2\pi \zeta \right|}^{2\alpha }{\left|\mathcal{F}\phi \left(\zeta \right)\right|}^2\mathrm{d}\zeta \hfill \\ & ={\int }_{\mathsf{\Omega }}{\left|{(-\mathsf{\Delta })}^{{\displaystyle \frac{\alpha }{2}}}\phi \right|}^2\mathrm{d}x=C(N,s){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}}\mathrm{d}x\mathrm{d}y,\hfill \end{array}$$

where $\mathcal{F}\phi \left(x\right)={\displaystyle \frac{1}{{\left(2\pi \right)}^{\frac{N}{2}}}}{\int }_{\mathsf{\Omega }}{e}^{-i\zeta x}\phi \left(x\right)\mathrm{d}x$ is the Fourier transform of $\phi$, see ([20], Propositions 3.4 and 3.6) and also Servadei-Valdinoci [22]. Hence, for $N>2\alpha$ and $\alpha \in (0,1)$, the fractional Hardy inequality is the following

$${\kappa }_0{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{\left|\phi \right|}^2}{{\left|x\right|}^{2\alpha }}}\mathrm{d}x\le {\int }_{\mathsf{\Omega }}{\left|2\pi \zeta \right|}^{2\alpha }{\left|\mathcal{F}\phi \left(\zeta \right)\right|}^2\mathrm{d}\zeta =C(N,s){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}}\mathrm{d}x\mathrm{d}y,$$

(5)

where $\phi \in C_0^{\infty }\left({\mathbb{R}}^N\right)$.

In the sequel, we will also use the space

$$X:=\left\{{\displaystyle \phi \in H^{\alpha }:\kappa {\int }_{\mathsf{\Omega }}\frac{\phi }{{\left|x\right|}^{2\alpha }}\mathrm{d}x<+\infty }\right\},$$

endowed with the above mentioned Gagliardo semi-norm, that is

$${\parallel \phi \parallel }_X={\left({\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}}\mathrm{d}x\mathrm{d}y\right)}^{{\displaystyle \frac{1}{2}}}.$$

Therefore, $(X,\parallel ·{\parallel }_X)$ is a Hilbert space with topological dual denoted by $X^{*}$, and the scalar product for $\phi ,u\in X$ is defined by

$$\langle \phi ,u\rangle ={\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{\left(\phi \right(x)-\phi (y\left)\right)\left(u\right(x)-u(y\left)\right)}{{|x-y|}^{N+2\alpha }}}\mathrm{d}x\mathrm{d}y.$$

Recall that $\phi \in X$ is a weak solution to (1) if

$$\begin{array}{cc}\hfill & {\displaystyle C(N,\alpha ){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{\left(\phi \right(x)-\phi (y\left)\right)\left(\varphi \right(x)-\varphi (y\left)\right)}{{|x-y|}^{N+2\alpha }}\mathrm{d}x\mathrm{d}y-\kappa {\int }_{\mathsf{\Omega }}\frac{\phi \varphi }{{\left|x\right|}^{2\alpha }}\mathrm{d}x}\hfill \\ \hfill =& {\displaystyle \frac{1}{2p}}{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\left|\phi \left(y\right)\right|}^p{\left|\phi \left(x\right)\right|}^{p-2}\phi \left(x\right)\varphi \left(x\right)}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y+{\displaystyle \frac{\lambda }{q}}{\int }_{\partial \mathsf{\Omega }}f\left(x\right){\displaystyle \frac{{\left|\phi \right|}^{q-1}\varphi }{{\left|x\right|}^{\beta }}}\mathrm{d}s,\hfill \end{array}$$

for all $\varphi \in X$.

Using the Hardy–Littlewood–Sobolev inequality

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{\tilde{g}\left(x\right)g\left(y\right)}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\le C(N,\tilde{\gamma },\mu ,\tilde{\vartheta })\parallel \tilde{g}{\parallel }_{\tilde{\gamma }}{\parallel g\parallel }_{\tilde{\vartheta }},\\ \hfill {\displaystyle \frac{1}{\tilde{\gamma }}}+{\displaystyle \frac{\mu }{N}}+{\displaystyle \frac{1}{\tilde{\vartheta }}}=2,\tilde{\gamma },\tilde{\vartheta },\tilde{g}\in L^{\tilde{\gamma }}\left({\mathbb{R}}^N\right),g\in L^{\tilde{\vartheta }}\left({\mathbb{R}}^N\right),\end{array}$$

(6)

we deduce that

$${\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}{\displaystyle \frac{{\left|\phi \left(x\right)\right|}^{2_{\mu ,\alpha }^{*}}{\left|\phi \left(y\right)\right|}^{2_{\mu ,\alpha }^{*}}}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\le C(N,\mu ){\parallel \phi \parallel }_{2_{\alpha }^{*}}^{{22}_{\mu ,\alpha }^{*}},$$

(7)

where $2_{\mu ,\alpha }^{*}=(2N-\mu )/(N-2\alpha )$. For $0\ne \phi \in X$, we define

$${\mathsf{\Lambda }}_0=\underset{\phi \in X\backslash \left\{0\right\}}{inf}{\displaystyle \frac{{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}\mathrm{d}x\mathrm{d}y}{{\left({\int }_{\mathsf{\Omega }}{\left|\phi \right|}^{2_{\alpha }^{*}}\mathrm{d}x\right)}^{\frac{2}{2_{\alpha }^{*}}}}},$$

and

$${\mathsf{\Lambda }}_1=\underset{\phi \in X\backslash \left\{0\right\}}{inf}{\displaystyle \frac{{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}\mathrm{d}x\mathrm{d}y}{{\left({\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{{\left|\phi \left(x\right)\right|}^{2_{\mu ,\alpha }^{*}}{\left|\phi \left(y\right)\right|}^{2_{\mu ,\alpha }^{*}}}{{|x-y|}^{\mu }}\mathrm{d}x\mathrm{d}y\right)}^{\frac{1}{2_{\mu ,\alpha }^{*}}}}}.$$

The fractional Hardy–Sobolev inequality is given in the following lemma ([23], see Lemma 2.1 of Ghoussoub-Shakerian).

Lemma 1.

If $0<\alpha <1$, $0\le \beta \le 2\alpha \le N$, and $1<q\le 2_{\beta ,\alpha }^{*}$, then we can find a positive constant $\tilde{C}>0$ satisfying

$${\displaystyle \tilde{C}{\left({\int }_{\partial \mathsf{\Omega }}\frac{{\left|\phi \right|}^q}{{\left|x\right|}^{\beta }}\mathrm{d}s\right)}^{\frac{2}{q}}\le C(N,\alpha ){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}\mathrm{d}x\mathrm{d}y-\kappa {\int }_{\mathsf{\Omega }}\frac{{\left|\phi \right|}^2}{{\left|x\right|}^{2\alpha }}\mathrm{d}x,\phi \in X,}$$

(8)

for $\kappa <{\kappa }_0$.

Further, the general best Hardy–Sobolev constant of (8) is defined by

$${\displaystyle \mathsf{\Xi }=\underset{\phi \in X,\phi \ne 0}{inf}\frac{{\displaystyle C(N,\alpha ){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}\mathrm{d}x\mathrm{d}y-\kappa {\int }_{\mathsf{\Omega }}\frac{{\left|\phi \right|}^2}{{\left|x\right|}^{2\alpha }}\mathrm{d}x}}{{\displaystyle {\left({\int }_{\partial \mathsf{\Omega }}\frac{{\left|\phi \right|}^q}{{\left|x\right|}^{\beta }}\mathrm{d}s\right)}^{\frac{2}{q}}}},}$$

(9)

where $1<q\le 2_{\beta ,\alpha }^{*}$ and $\kappa <{\kappa }_0$.

We note that the Hardy inequality (5) yields that the space X is continuously embedded in the weighted space $L^2{(\mathsf{\Omega },|x|}^{-2\alpha })$. If $\kappa <{\kappa }_0$, from (5) we consider the norm

$$\parallel \phi \parallel :={\left({\displaystyle C(N,\alpha ){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}\frac{{|\phi \left(x\right)-\phi \left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}\mathrm{d}x\mathrm{d}y-\kappa {\int }_{\mathsf{\Omega }}\frac{{\left|\phi \right|}^2}{{\left|x\right|}^{2\alpha }}\mathrm{d}x}\right)}^{{\displaystyle \frac{1}{2}}},$$

which is well defined on X and equivalent to the norm ${\parallel ·\parallel }_X$. From Fan, Zhao [24], Deng [25], and Chen et al. [26], we recall some useful embedding results.

Lemma 2.

([24]). The following assertions are valid:

• The embedding $X\hookrightarrow L^r\left(\mathsf{\Omega }\right)$ is continuous for $r\in [1,2_{\alpha }^{*}]$;
• The embedding $X\hookrightarrow L^r\left(\mathsf{\Omega }\right)$ is compact for $r\in [1,2_{\alpha }^{*})$.

Lemma 3.

([25,26]). Set $2_{b,\alpha }^{*}={\displaystyle \frac{2(N-b)}{N-2\alpha }}$. The following assertions are valid:

• The embedding $X\hookrightarrow L^{\gamma }{(\partial \mathsf{\Omega },|x|}^{-\beta })$ is continuous for $\gamma \in (0,2_{\beta ,\alpha }^{*}]$;
• Suppose that $0<b<2\alpha$. If $1\le \gamma <2_{b,\alpha }^{*}$ and $0\le \beta <\alpha \gamma +N(1-{\displaystyle \frac{\gamma }{2}})$, the embedding $X\hookrightarrow L^{\gamma }{(\partial \mathsf{\Omega },|x|}^{-\beta })$ is compact.

The starting point to study weak solutions of problem (1) is the analysis of the associated functional defined by

$$\begin{array}{cc}\hfill {\mathcal{J}}_{\lambda }\left(\phi \right):=& {\displaystyle \frac{1}{2}}{\parallel \phi \parallel }^2-{\displaystyle \frac{1}{2p}}{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\left|\phi \left(x\right)\right|}^p{\left|\phi \left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y-{\displaystyle \frac{\lambda }{q}}{\int }_{\partial \mathsf{\Omega }}f\left(x\right){\displaystyle \frac{{\left|\phi \right|}^q}{{\left|x\right|}^{\beta }}}\mathrm{d}s\hfill \\ \hfill =& {\displaystyle \frac{1}{2}}{\parallel \phi \parallel }^2-{\displaystyle \frac{1}{2p}}P\left(\phi \right)-{\displaystyle \frac{\lambda }{q}}K\left(\phi \right),\hfill \end{array}$$

where

$$P\left(\phi \right)={\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\left|\phi \left(x\right)\right|}^p{\left|\phi \left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y,K\left(\phi \right)={\int }_{\partial \mathsf{\Omega }}f\left(x\right){\displaystyle \frac{{\left|\phi \right|}^q}{{\left|x\right|}^{\beta }}}\mathrm{d}s.$$

Note that $(2N-\mu )/2N<p<2_{\mu ,\alpha }^{*}$, then the Hardy–Littlewood–Sobolev inequality, together with the Hölder inequality, give us the a priori estimate

$$\begin{array}{cc}\hfill P\left(\phi \right)& ={\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\left|\phi \left(x\right)\right|}^p{\left|\phi \left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & \le {C(N,\mu ,p)\parallel h\parallel }_{\infty }{\parallel \phi \parallel }_{p\vartheta }^{2p}\hfill \\ \hfill & \le {C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\parallel \phi \parallel }_{2_{\alpha }^{*}}^{2p}\hfill \\ \hfill & \le {C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}{\parallel \phi \parallel }^{2p},\hfill \end{array}$$

(10)

where $\vartheta =\left(2N\right)/(2N-\mu )$, $p\vartheta <2_{\alpha }^{*}$.

3. Nehari Manifold

In this section, we establish several preparatory results under the same assumptions as in Theorems 1.1 and 1.2, namely, $1<2p<2<q$. For every $\lambda >0$, we introduce the Nehari manifold for the energy functional associated with problem (1) defined by

$${\mathcal{Q}}_{\lambda }=\{\phi \in X\backslash \left\{0\right\}:\langle {\mathcal{J}}_{\lambda }^{\prime }\left(\phi \right),\phi \rangle =0\},$$

and for the related minimization problem, we set

$${\beta }_0:=inf\{{\mathcal{J}}_{\lambda }\left(\phi \right):\phi \in {\mathcal{Q}}_{\lambda }\},$$

where $\langle ·,·\rangle$ denotes the duality brackets for the pair $(X^{*},X)$. Evidently, every critical point of ${\mathcal{J}}_{\lambda }$ is contained in ${\mathcal{Q}}_{\lambda }$; hence, the condition $\phi \in {\mathcal{Q}}_{\lambda }$ equivalently states

$$\langle {\mathcal{J}}_{\lambda }^{\prime }\left(\phi \right),\phi \rangle ={\parallel \phi \parallel }^2-P\left(\phi \right)-\lambda K\left(\phi \right)=0.$$

(11)

We set

$${\varphi }_{\phi }\left(\phi \right):=\langle {\mathcal{J}}_{\lambda }^{\prime }\left(\phi \right),\phi \rangle ,$$

so that we get

$$\begin{array}{cc}\hfill \langle {\varphi }_{\phi }^{\prime }\left(\phi \right),\phi \rangle =& {2\parallel \phi \parallel }^2-2pP\left(\phi \right)-\lambda qK\left(\phi \right)\hfill \\ \hfill =& (2-2p)P\left(\phi \right)+\lambda (2-q)K\left(\phi \right)\hfill \\ \hfill =& {(2-2p)\parallel \phi \parallel }^2-\lambda (q-2p)K\left(\phi \right)\hfill \\ \hfill =& {(2-q)\parallel \phi \parallel }^2-(2p-q)P\left(\phi \right).\hfill \end{array}$$

(12)

In the sequel, it is helpful to decompose the Nehari manifold ${\mathcal{Q}}_{\lambda }$ into three submanifolds, corresponding to local minima, local maxima, and points of inflection, that is

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{\lambda }^{+}& =\{\phi \in {\mathcal{Q}}_{\lambda }:\langle {\varphi }_{\phi }^{\prime }\left(\phi \right),\phi \rangle >0\};\hfill \\ \hfill {\mathcal{Q}}_{\lambda }^{-}& =\{\phi \in {\mathcal{Q}}_{\lambda }:\langle {\varphi }_{\phi }^{\prime }\left(\phi \right),\phi \rangle <0\};\hfill \\ \hfill {\mathcal{Q}}_{\lambda }^0& =\{\phi \in {\mathcal{Q}}_{\lambda }:\langle {\varphi }_{\phi }^{\prime }\left(\phi \right),\phi \rangle =0\}.\hfill \end{array}$$

(13)

Lemma 4.

If φ is a local minimizer of ${\mathcal{J}}_{\lambda }$ on ${\mathcal{Q}}_{\lambda }$ and $\langle {\varphi }^{\prime }\left(\phi \right),\phi \rangle \ne 0$, then ${\mathcal{J}}_{\lambda }^{\prime }\left(\phi \right)=0$ in $X^{*}$.

Proof. 

The proof is similar to that of Brown and Zhang ([17], Theorem 2.3), and then we omit the details. □

Here, we revisit the definition of the Palais–Smale condition at level c.

Definition 1.

Let $\tilde{h}\left(x\right)\in C^1(X,\mathbb{R})$ and $c\in \mathbb{R}$. The function $\tilde{h}\left(x\right)$ satisfies the ${\left(PS\right)}_c$-condition if any sequence $\left\{{\phi }_n\right\}\subset X$ such that

$$\tilde{h}\left(x\right)\left({\phi }_n\right)=c+o\left(1\right)\mathrm{and}\tilde{h}{\left(x\right)}^{\prime }\left({\phi }_n\right)=o\left(1\right)\mathrm{in}{X}^{-1}\mathrm{as}n\to \infty ,$$

admits a convergent subsequence.

Lemma 5.

There exists a constant ${\lambda }_0>0$ such that ${\mathcal{Q}}_{\lambda }^0=\varnothing$ for all $0<\lambda <{\lambda }_0$.

Proof. 

We argue by contradiction, and suppose that ${\mathcal{Q}}_{\lambda }^0\ne \varnothing$ for

$$0<\lambda <{\lambda }_0={\left({\displaystyle \frac{2p-q}{2-q}}{C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}\right)}^{{\displaystyle \frac{2-q}{2-2p}}}{\displaystyle \frac{2-2p}{q-2p}}{\mathcal{A}}^{-1}{\mathsf{\Xi }}^{{\displaystyle \frac{q}{2}}},$$

(14)

where $\mathcal{A}={\parallel f\parallel }_{\infty }$. Then, for $\phi \in {\mathcal{Q}}_{\lambda }^0$ and (12), we have

$$\begin{array}{c}\hfill 0=\langle {\varphi }_{\phi }^{\prime }\left(\phi \right),\phi \rangle =(2-2p){\parallel \phi \parallel }^2-\lambda (q-2p)K\left(\phi \right).\end{array}$$

(15)

Combining (9) and (15), we obtain

$$\begin{array}{c}\hfill {\parallel \phi \parallel }^2=\lambda {\displaystyle \frac{q-2p}{2-2p}}K\left(\phi \right)=\lambda {\displaystyle \frac{q-2p}{2-2p}}{\int }_{\partial \mathsf{\Omega }}f\left(x\right){\displaystyle \frac{{\left|\phi \right|}^q}{{\left|x\right|}^{\beta }}}\mathrm{d}s\le \lambda {\displaystyle \frac{q-2p}{2-2p}}\mathcal{A}{\mathsf{\Xi }}^{-{\displaystyle \frac{q}{2}}}{\parallel \phi \parallel }^q,\end{array}$$

(16)

which shows

$${\left(\lambda {\displaystyle \frac{q-2p}{2-2p}}\mathcal{A}{\mathsf{\Xi }}^{-{\displaystyle \frac{q}{2}}}\right)}^{-1}\le {\parallel \phi \parallel }^{q-2}.$$

Then,

$${\left({\displaystyle \frac{2-2p}{\lambda (q-2p)}}{\mathcal{A}}^{-1}{\mathsf{\Xi }}^{{\displaystyle \frac{q}{2}}}\right)}^{{\displaystyle \frac{1}{q-2}}}\le \parallel \phi \parallel .$$

(17)

Combining (10) and (15), we obtain

$$\begin{array}{cc}\hfill {(2-q)\parallel \phi \parallel }^2=& (2p-q)P\left(\phi \right)=(2p-q){\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\left|\phi \left(x\right)\right|}^p{\left|\phi \left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & \ge {(2p-q)C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}{\parallel \phi \parallel }^{2p},\hfill \end{array}$$

(18)

which means

$${\parallel \phi \parallel }^2\le {\displaystyle \frac{2p-q}{2-q}}{C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}{\parallel \phi \parallel }^{2p}.$$

(19)

Therefore, we deduce that

$$\parallel \phi \parallel \le {\left({\displaystyle \frac{2p-q}{2-q}}{C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}\right)}^{{\displaystyle \frac{1}{2-2p}}}.$$

(20)

Now, combining the estimates (17) and (20), we obtain $\lambda \ge {\lambda }_0$, which leads to a contradiction with the initial assumption (14) on $\lambda$. Hence, there exists a constant ${\lambda }_0>0$ such that ${\mathcal{Q}}_{\lambda }=\varnothing$ whenever $0<\lambda <{\lambda }_0$. □

Now, we establish the coercivity of the functional on the Nehari manifold (that is, ${\mathcal{J}}_{\lambda }\left(\phi \right)\to \infty$ as $\parallel \phi \parallel \to \infty$).

Lemma 6.

${\mathcal{J}}_{\lambda }$ is bounded below and coercive on ${\mathcal{Q}}_{\lambda }$.

Proof. 

For $1<2p<2<q$, we deduce by (11) that

$$\begin{array}{cc}\hfill {\mathcal{J}}_{\lambda }\left(\phi \right)=& {\displaystyle \frac{1}{2}}{\parallel \phi \parallel }^2-{\displaystyle \frac{1}{2p}}P\left(\phi \right)-{\displaystyle \frac{\lambda }{q}}K\left(\phi \right)\hfill \\ \hfill =& ({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{q}}){\parallel \phi \parallel }^2+({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}})P\left(\phi \right)\hfill \\ \hfill \ge & ({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{q}}){\parallel \phi \parallel }^2+({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}})C(N,\mu ,p){\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}{\parallel \phi \parallel }^{2p},\hfill \end{array}$$

which implies that ${\mathcal{J}}_{\lambda }\left(\phi \right)$ is bounded below on ${\mathcal{Q}}_{\lambda }$. From the last inequality and since $1<2p<2<q$, we can conclude that ${\mathcal{J}}_{\lambda }$ is coercive on ${\mathcal{Q}}_{\lambda }$. □

For $0<\lambda <{\lambda }_0$, by Lemmas 5 and 6, we deduce that ${\mathcal{Q}}_{\lambda }={\mathcal{Q}}_{\lambda }^{+}\cup {\mathcal{Q}}_{\lambda }^{-}$ and ${\mathcal{J}}_{\lambda }$ is bounded from below on ${\mathcal{Q}}_{\lambda }^{+}$ and ${\mathcal{Q}}_{\lambda }^{-}$. Thus, we set

$${\beta }_0^{+}=\underset{\phi \in {\mathcal{Q}}_{\lambda }^{+}}{inf}{\mathcal{J}}_{\lambda }\left(\phi \right),{\beta }_0^{-}=\underset{\phi \in {\mathcal{Q}}_{\lambda }^{-}}{inf}{\mathcal{J}}_{\lambda }\left(\phi \right).$$

Then, we have the following results.

Lemma 7.

The following assertions hold:

(i) 

If $0<\lambda <{\lambda }_0$, then ${\beta }_0\le {\beta }_0^{+}<0$;

(ii) 

If $0<\lambda <{\overline{\lambda }}_0$, then ${\beta }_0^{-}>{\beta }_{*}>0$ for some ${\beta }_{*}>0$.

Proof. 

(i) For $\phi \in {\mathcal{Q}}_{\lambda }^{+}$, we have

$${\displaystyle \frac{2-2p}{q-2p}}{\parallel \phi \parallel }^2>\lambda K\left(\phi \right),$$

which gives us

$$\begin{array}{cc}\hfill {\mathcal{J}}_{\lambda }\left(\phi \right)=& {\displaystyle \frac{1}{2}}{\parallel \phi \parallel }^2-{\displaystyle \frac{1}{2p}}P\left(\phi \right)-{\displaystyle \frac{\lambda }{q}}K\left(\phi \right)\hfill \\ \hfill =& ({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2p}}){\parallel \phi \parallel }^2+\left({\displaystyle \frac{1}{2p}}-{\displaystyle \frac{1}{q}}\right)\lambda K\left(\phi \right)\hfill \\ \hfill \le & \left({\displaystyle \frac{p-1}{2p}}+{\displaystyle \frac{2-2p}{2pq}}\right){\parallel \phi \parallel }^2\hfill \\ \hfill =& {\displaystyle \frac{(q-2)(p-1)}{2pq}}{\parallel \phi \parallel }^2<0.\hfill \end{array}$$

This shows that ${\beta }_0\le {\beta }_0^{+}<0$.

(ii) For $\phi \in {\mathcal{Q}}_{\lambda }^{-}$, we have

$${\parallel \phi \parallel }^2<\lambda {\displaystyle \frac{q-2p}{2-2p}}K\left(\phi \right),$$

which implies

$$\parallel \phi \parallel >{\left({\displaystyle \frac{2-2p}{\lambda (q-2p)}}{\mathcal{A}}^{-1}{\mathsf{\Xi }}^{{\displaystyle \frac{q}{2}}}\right)}^{{\displaystyle \frac{1}{q-2}}}.$$

Combining (10) and (11), we obtain

$$\begin{array}{cc}\hfill {\mathcal{J}}_{\lambda }\left(\phi \right)& ={\displaystyle \frac{1}{2}}{\parallel \phi \parallel }^2-{\displaystyle \frac{1}{2p}}P\left(\phi \right)-{\displaystyle \frac{\lambda }{q}}K\left(\phi \right)\hfill \\ \hfill & \ge \left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{q}}\right){\parallel \phi \parallel }^2+\left({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}}\right){C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}{\parallel \phi \parallel }^{2p}\hfill \\ \hfill & ={\parallel \phi \parallel }^{2p}\left(\left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{q}}\right){\parallel \phi \parallel }^{2-2p}+\left({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}}\right){C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}\right)\hfill \\ \hfill & {>\parallel \phi \parallel }^{2p}\left(\left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{q}}\right){\left({\displaystyle \frac{2-2p}{\lambda (q-2p)}}{\mathcal{A}}^{-1}{\mathsf{\Xi }}^{{\displaystyle \frac{q}{2}}}\right)}^{{\displaystyle \frac{2-2p}{q-2}}}\right.\hfill \\ \hfill & \left.+\left({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}}\right){C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}\right).\hfill \end{array}$$

If $0<\lambda <{\lambda }_0$, then we obtain

$${\mathcal{J}}_{\lambda }\left(\phi \right)>{\beta }_{*},\mathrm{for}\mathrm{all}\phi \in {\mathcal{Q}}_{\lambda }^{-},$$

for some constant ${\beta }_{*}={\beta }_{*}(N,\mu ,p,q,\vartheta ,\alpha ,|\mathsf{\Omega }|,{\mathsf{\Lambda }}_0)$, and so the proof of assertion (ii) is established. □

Lemma 8.

Let $\tilde{\lambda }={\mathcal{A}}^{-1}{\mathsf{\Xi }}^{{\displaystyle \frac{q}{2}}}\left({\displaystyle \frac{2-2p}{q-2p}}\right){\left({\displaystyle \frac{q-2}{q-2p}}\right)}^{{\displaystyle \frac{q-2}{2-2p}}}{\left({C(N,\mu ,p)\parallel h\parallel }_{\infty }{\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\mathsf{\Lambda }}_0^{-p}\right)}^{{\displaystyle \frac{q-2}{2p-2}}}$. Then, for all $\phi \in X\backslash \left\{0\right\}$ and $\lambda \in (0,\tilde{\lambda })$, there exist unique $t^{+}=t^{+}\left(\phi \right)>0$ and $t^{-}=t^{-}\left(\phi \right)>0$ such that $t^{+}\phi \in {\mathcal{Q}}_{\lambda }^{+}$ and $t^{-}\phi \in {\mathcal{Q}}_{\lambda }^{-}$ and ${\mathcal{Q}}_{\lambda }^{-}=\{\phi \in X\backslash \left\{0\right\}:t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)=\parallel \phi \parallel \}$. We have

$$t^{-}>t_{max}=t_0={\left({\displaystyle \frac{{(2-2p)\parallel \phi \parallel }^2}{(q-2p)\lambda K\left(\phi \right)}}\right)}^{{\displaystyle \frac{1}{q-2}}}>t^{+}\ge 0,$$

and

$${\mathcal{J}}_{\lambda }\left(t^{+}\phi \right)=\underset{0\le t\le t^{-}\left(\phi \right)}{min}{\mathcal{J}}_{\lambda }\left(t\phi \right),\mathcal{J}\left(t^{-}\phi \right)=\underset{t>t_0}{max}\mathcal{J}\left(t\phi \right).$$

Proof. 

Using (11) and (12), we obtain

$$\psi \left(t\right):=t^{2-2p}{\parallel \phi \parallel }^2-t^{q-2p}\lambda K\left(\phi \right).$$

Clearly, for $t>0$, $t\phi \in {\mathcal{Q}}_{\lambda }$ if and only if t is the solution of the equation

$$\psi \left(t\right)=P\left(\phi \right).$$

Since $q-2p>2-2p>0$, we know that function $\psi \left(t\right)$ is initially increasing and eventually decreasing with a single turning point $t_0={\left({\displaystyle \frac{{(2-2p)\parallel \phi \parallel }^2}{(q-2p)\lambda K\left(\phi \right)}}\right)}^{{\displaystyle \frac{1}{q-2}}}$, that is, for the following equation

$${\psi }^{\prime }\left(t\right)=(2-2p)t^{1-2p}{\parallel \phi \parallel }^2-(q-2p)\lambda t^{q-2p-1}K\left(\phi \right),$$

there is ${\psi }^{\prime }\left(t_0\right)=0$, ${\psi }^{\prime }\left(t\right)>0$ for $t\in [0,t_0)$ and ${\psi }^{\prime }\left(t\right)<0$ for $t\in (t_0,+\infty )$. Moreover, by (10), we get

$$\begin{array}{cc}\hfill \psi \left(t_0\right)=& \left({\displaystyle \frac{q-2}{q-2p}}\right){\left({\displaystyle \frac{2-2p}{q-2p}}\right)}^{{\displaystyle \frac{2-2p}{q-2}}}{\left({\displaystyle \frac{{\parallel \phi \parallel }^{2(q-2p)}}{{\left(\lambda K\left(\phi \right)\right)}^{2-2p}}}\right)}^{{\displaystyle \frac{1}{q-2}}}\hfill \\ \hfill \ge & \left({\displaystyle \frac{q-2}{q-2p}}\right){\left({\displaystyle \frac{2-2p}{q-2p}}\right)}^{{\displaystyle \frac{2-2p}{q-2}}}{\parallel \phi \parallel }^{2p}{\left(\lambda \mathcal{A}{\mathsf{\Xi }}^{-{\displaystyle \frac{q}{2}}}\right)}^{-{\displaystyle \frac{2-2p}{q-2}}}.\hfill \end{array}$$

(21)

We now distinguish the cases when $P\left(\phi \right)\le 0$ and $P\left(\phi \right)>0$.

(i)

If $P\left(\phi \right)\le 0$, then we can find a unique $t^{-}:=t^{-}\left(\phi \right)>t_0$ such that

$$\psi \left(t^{-}\right)=P\left(\phi \right)and{\psi }^{\prime }\left(t^{-}\right)<0.$$

(22)

We claim that $t^{-}\phi \in {\mathcal{Q}}_{\lambda }^{-}$. Clearly, from $t^{-}\phi \in X$ and above equation, we have

$$\begin{array}{cc}\hfill \langle {\mathcal{J}}_{\lambda }^{\prime }\left(t^{-}\phi \right),t^{-}\phi \rangle & =\parallel t^{-}{\phi \parallel }^2-P\left(t^{-}\phi \right)-\lambda K\left(t^{-}\phi \right)\hfill \\ \hfill & ={\left(t^{-}\right)}^{2p}\left({\left(t^{-}\right)}^{2-2p}{\parallel \phi \parallel }^2-\lambda {\left(t^{-}\right)}^{q-2p}K\left(\phi \right)-P\left(\phi \right)\right)\hfill \\ \hfill & ={\left(t^{-}\right)}^{2p}\left(\psi \left(t^{-}\right)-P\left(\phi \right)\right)=0,\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \langle {\varphi }_{\phi }^{\prime }\left(t^{-}\phi \right),t^{-}\phi \rangle & =2\parallel t^{-}{\phi \parallel }^2-2pP\left(t^{-}\phi \right)-q\lambda K\left(t^{-}\phi \right)\hfill \\ \hfill & =(2-2p)\parallel t^{-}{\phi \parallel }^2-(q-2p)\lambda K\left(t^{-}\phi \right)\hfill \\ \hfill & ={\left(t^{-}\right)}^{2p+1}\left((2-2p){\left(t^{-}\right)}^{1-2p}{\parallel \phi \parallel }^2-\lambda (q-2p){\left(t^{-}\right)}^{q-2p-1}K\left(\phi \right)\right)\hfill \\ \hfill & ={\left(t^{-}\right)}^{2p+1}{\psi }^{\prime }\left(t^{-}\right)<0,\hfill \end{array}$$

which show that $t^{-}\phi \in {\mathcal{Q}}_{\lambda }^{-}$. Next, we prove that ${\mathcal{J}}_{\lambda }\left(t^{-}\phi \right)={max}_{t\ge t_{max}}{\mathcal{J}}_{\lambda }\left(t\phi \right)$. It follows from (22) that

$$\begin{array}{cc}\hfill {\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}}{\mathcal{J}}_{\lambda }\left(t^{-}\phi \right)& =t^{-}{\parallel \phi \parallel }^2-{\left(t^{-}\right)}^{2p-1}P\left(\phi \right)-{\left(t^{-}\right)}^{q-1}\lambda K\left(\phi \right)\hfill \\ \hfill & ={\left(t^{-}\right)}^{2p-1}\left({\left(t^{-}\right)}^{2-2p}{\parallel \phi \parallel }^2-P\left(\phi \right)-\lambda {\left(t^{-}\right)}^{q-2p-1}K\left(\phi \right)\right)\hfill \\ \hfill & ={\left(t^{-}\right)}^{2p-1}\left(\psi \left(t^{-}\right)-P\left(\phi \right)\right)=0,\hfill \end{array}$$

(23)

and

$$\begin{array}{cc}\hfill t^2{\displaystyle \frac{{\mathrm{d}}^2}{\mathrm{d}{t}^2}}{\mathcal{J}}_{\lambda }\left(t^{-}\phi \right)=& \parallel t^{-}{\phi \parallel }^2-(2p-1)P\left(t^{-}\phi \right)-(q-1)\lambda K\left(t^{-}\phi \right)\hfill \\ \hfill =& {\left(t^{-}\right)}^{2p+1}\left((2-2p){\left(t^{-}\right)}^{1-2p}{\parallel \phi \parallel }^2-\lambda (q-2p){\left(t^{-}\right)}^{q-2p-1}K\left(\phi \right)\right)\hfill \\ \hfill =& {\left(t^{-}\right)}^{2p+1}{\psi }^{\prime }\left(t^{-}\right)<0,\mathrm{for}\mathrm{all}t>t_{max}.\hfill \end{array}$$

(24)

This proves the claim. Now, we consider the other case.

(ii)

If $P\left(\phi \right)>0$, then from (21), we obtain

$$\begin{array}{cc}\hfill 0=\psi \left(0\right)& <P\le {C(N,\mu ,p)\left|\mathsf{\Omega }\right|}^{{\displaystyle \frac{2(2_{\alpha }^{*}-p\vartheta )}{2_{\alpha }^{*}\vartheta }}}{\parallel h\parallel }_{\infty }{\mathsf{\Lambda }}_0^{-p}{\parallel \phi \parallel }^{2p}\hfill \\ \hfill & \le \left({\displaystyle \frac{q-2}{q-2p}}\right){\left({\displaystyle \frac{2-2p}{q-2p}}\right)}^{{\displaystyle \frac{2-2p}{q-2}}}{\parallel \phi \parallel }^{2p}{\left(\lambda \mathcal{A}{\mathsf{\Xi }}^{-{\displaystyle \frac{q}{2}}}\right)}^{-{\displaystyle \frac{2-2p}{q-2}}}\hfill \\ \hfill & \le \psi \left(t_0\right)\mathrm{for}0<\lambda <\tilde{\lambda }.\hfill \end{array}$$

For $K\left(\phi \right)>0$ and $\psi \left(t_0\right)>0$, there exist unique $t^{+}$ and $t^{-}$ such that $0<t^{+}<t_0<t^{-}$, and

$$\psi \left(t^{+}\right)=\lambda K\left(\phi \right)=\psi \left(t^{-}\right),and\psi \left(t^{+}\right)>0>\psi \left(t^{-}\right).$$

We have $t^{-}\phi \in {\mathcal{Q}}_{\lambda }^{-}$, $t^{+}\phi \in {\mathcal{Q}}_{\lambda }^{+}$, and ${\mathcal{J}}_{\lambda }\left(t^{-}\phi \right)\ge {\mathcal{J}}_{\lambda }\left(t\phi \right)\ge {\mathcal{J}}_{\lambda }\left(t^{+}\phi \right)$ for $t\in [t^{+},t^{-}]$ and ${\mathcal{J}}_{\lambda }\left(t^{+}\phi \right)\le {\mathcal{J}}_{\lambda }\left(t\phi \right)$ for $t\in [0,t^{+}]$. Hence, we deduce that

$${\mathcal{J}}_{\lambda }\left(t^{+}\phi \right)=\underset{0\le t\le t^{-}}{min}{\mathcal{J}}_{\lambda }\left(t\phi \right),{\mathcal{J}}_{\lambda }\left(t^{-}\phi \right)=\underset{t>t_0}{max}{\mathcal{J}}_{\lambda }\left(t\phi \right).$$

Therefore, the claim is proved. □

4. Proof of Theorem 1

In this section, using the Nehari method, we establish our first result, namely Theorem 1, which says that for sufficiently small $\lambda >0$, problem (1) admits at least two positive solutions in the subcritical case $q<2_{\beta ,\alpha }^{*}$. More precisely, our proof will be divided into two lemmas, but before we note the following proposition about the existence of Palais–Smale sequences at level ${\beta }_0^{\pm }$ for the functional ${\mathcal{J}}_{\lambda }$.

Proposition 1.

If $0<\lambda <min\{{\lambda }_0,\tilde{\lambda }\}$, then the following assertions hold:

(i) 

There exists a ${\left(PS\right)}_{{\beta }_0^{+}}$-sequence $\left\{{\phi }_n\right\}\subset {\mathcal{Q}}_{\lambda }$ in X for ${\mathcal{J}}_{\lambda }$;

(ii) 

There exists a ${\left(PS\right)}_{{\beta }_0^{-}}$-sequence $\left\{{\phi }_n\right\}\subset {\mathcal{Q}}_{\lambda }^{-}$ in X for ${\mathcal{J}}_{\lambda }$.

The proof of Proposition 1 can be concluded by adapting the steps in the proof by Wu ([27], Proposition 9), and so the details are omitted. Now, we discuss the existence of local minimizers to the energy functional.

Lemma 9.

If $0<\lambda <min\{{\lambda }_0,\tilde{\lambda }\}$, then the functional ${\mathcal{J}}_{\lambda }$ admits a minimizer ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$, satisfying the following conditions:

• ${\mathcal{J}}_{\lambda }\left({\phi }_1\right)={\beta }_0={\beta }_0^{+}<0$;
• ${\phi }_1$ is a positive solution to problem (1).

Proof. 

By Proposition 1, one can find a minimizing sequence $\left\{{\phi }_n\right\}$ for ${\mathcal{J}}_{\lambda }$ on ${\mathcal{Q}}_{\lambda }$ such that

$${\mathcal{J}}_{\lambda }\left({\phi }_n\right)={\beta }_0+o\left(1\right)\mathrm{and}{\mathcal{J}}_{\lambda }^{\prime }\left({\phi }_n\right)=o\left(1\right)\mathrm{in}{X}^{*}.$$

(25)

Since the functional ${\mathcal{J}}_{\lambda }$ is bounded from below on the Nehari manifold ${\mathcal{Q}}_{\lambda }$, there exists a minimizing sequence $\left\{{\phi }_n\right\}\subseteq {\mathcal{Q}}_{\lambda }$ such that, passing to the limit, we have

$$\underset{n\to \infty }{lim}{\mathcal{J}}_{\lambda }\left({\phi }_n\right)=\underset{\phi \in {\mathcal{Q}}_{\lambda }}{inf}{\mathcal{J}}_{\lambda }\left(\phi \right).$$

Lemma 6 ensures that the sequence $\left\{{\phi }_n\right\}$ is bounded in X. So, using the embeddings results in Lemmas 2 and 3, we may assume that

$${\phi }_n\rightharpoonup {\phi }_1\mathrm{in}X,{\phi }_n\to {\phi }_1\mathrm{in}{L}^r\left(\mathsf{\Omega }\right)\mathrm{and}{L}^{\gamma }(\partial \mathsf{\Omega }),$$

for some ${\phi }_1\in X$, $1\le \gamma <min\{2_{\beta ,\alpha }^{*},2_{b,\alpha }^{*}\}$, $1\le r<2_{\alpha }^{*}$, $0<b<2\alpha$, and $0\le \beta <\alpha \gamma +N(1-{\displaystyle \frac{\gamma }{2}})$. Hence, we easily get

$${\int }_{\mathsf{\Omega }}|{\phi }_n|\mathrm{d}x\to {\int }_{\mathsf{\Omega }}|{\phi }_1|\mathrm{d}x\mathrm{and}{\int }_{\partial \mathsf{\Omega }}|{\phi }_n|\mathrm{d}s\to {\int }_{\partial \mathsf{\Omega }}\left|{\phi }_1\right|\mathrm{d}s,$$

(26)

as $n\to \infty$. Then,

$$K\left({\phi }_n\right)=K\left({\phi }_1\right)+o\left(1\right)(n\to \infty ).$$

(27)

By (25) and (26), ${\phi }_1$ is a weak solution of problem (1). Using the definition of ${\mathcal{J}}_{\lambda }$ and (11), we have

$$K\left({\phi }_n\right)={\displaystyle \frac{q(p-1)}{2p-q}}{\parallel {\phi }_n\parallel }^2-{\displaystyle \frac{2pq}{2p-q}}{\mathcal{J}}_{\lambda }\left({\phi }_n\right).$$

(28)

For $n\to \infty$ in (28), combining (25), (27), and ${\beta }_0<0$, we obtain

$$K\left({\phi }_1\right)\ge -{\displaystyle \frac{2pq}{2p-q}}{\beta }_0>0.$$

Hence, ${\phi }_1\in {\mathcal{Q}}_{\lambda }$ is a nontrivial solution of problem (1).

Now, we prove that ${\phi }_n\to {\phi }_1$ in X. By the Fatou Lemma, we get

$$\begin{array}{cc}\hfill {\beta }_0\le {\mathcal{J}}_{\lambda }\left({\phi }_1\right)=& \left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2p}}\right){\parallel {\phi }_1\parallel }^2-\left({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}}\right)\lambda K\left({\phi }_1\right)\hfill \\ \hfill \le & \underset{n\to \infty }{lim}inf\left(\left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2p}}\right){\parallel {\phi }_n\parallel }^2-\left({\displaystyle \frac{1}{q}}-{\displaystyle \frac{1}{2p}}\right)\lambda K\left({\phi }_n\right)\right)={\beta }_0,\hfill \end{array}$$

which yields ${\mathcal{J}}_{\lambda }\left({\phi }_1\right)={\beta }_0$, and ${\phi }_n\to {\phi }_1$ strongly in X.

Next, we have to prove that ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$. Suppose that ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{-}$. Utilizing Lemma 8, we can find $t^{-}$ and $t^{+}$ such that $t^{-}{\phi }_1\in {\mathcal{Q}}_{\lambda }^{-}$, $t^{+}{\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$ and $t^{+}<t^{-}=1$. Now we have ${\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}}{\mathcal{J}}_{\lambda }=0$ and ${\displaystyle \frac{{\mathrm{d}}^2}{\mathrm{d}{t}^2}}\mathcal{J}>0$. Consequently, there exists $\tilde{t}$ such that $t^{+}<\tilde{t}<t^{-}=1$ and ${\mathcal{J}}_{\lambda }\left(t^{+}{\phi }_1\right)<{\mathcal{J}}_{\lambda }\left(\tilde{t}{\phi }_1\right)$. From Lemma 8, we also have

$${\mathcal{J}}_{\lambda }\left(t^{+}{\phi }_1\right)<{\mathcal{J}}_{\lambda }\left(\tilde{t}{\phi }_1\right)\le {\mathcal{J}}_{\lambda }\left(t^{-}{\phi }_1\right)={\mathcal{J}}_{\lambda }\left({\phi }_1\right),$$

this leads to a contradiction with ${\mathcal{J}}_{\lambda }\left({\phi }_1\right)={\beta }_0$. So, we obtain that ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$, and ${\mathcal{J}}_{\lambda }\left({\phi }_1\right)={\beta }_0^{+}={\beta }_0$. Clearly, we have ${\mathcal{J}}_{\lambda }\left({\phi }_1\right)={\mathcal{J}}_{\lambda }\left(\right|{\phi }_1\left|\right)$, and $|{\phi }_1|\in {\mathcal{Q}}^{+}$, and hence it solves problem (1). From Lemma 4, we can assume that ${\phi }_1\ge 0$. Finally, by the strong maximum principle (see [27]), we conclude that this solution is positive, namely ${\phi }_1>0$. □

Lemma 10.

If $0<\lambda <\overline{\lambda }=min\{{\lambda }_0,\tilde{\lambda }\}$, then the functional ${\mathcal{J}}_{\lambda }$ admits a minimizer ${\phi }_2\in {\mathcal{Q}}_{\lambda }^{-}$ satisfying the following conditions:

• ${\mathcal{J}}_{\lambda }\left({\phi }_2\right)={\beta }_0^{-}$;
• ${\phi }_2$ is a positive solution to problem (1).

Proof. 

Proposition 1 implies that we can find a minimizing sequence $\left\{{\phi }_n\right\}$ of the functional ${\mathcal{J}}_{\lambda }$ on the submanifold ${\mathcal{Q}}^{-}$ satisfying the following conditions

$${\mathcal{J}}_{\lambda }\left({\phi }_n\right)={\beta }_0^{-}+o\left(1\right)\mathrm{and}{\mathcal{J}}_{\lambda }^{\prime }=\left({\phi }_n\right)in{X}^{*}.$$

(29)

The sequence $\left\{{\phi }_n\right\}$ is bounded in X by Lemma 6. So, from Lemmas 2 and 3, we may suppose there exists ${\phi }_2\in X$ such that

$${\phi }_n\rightharpoonup {\phi }_2\mathrm{in}X,{\phi }_n\to {\phi }_2\mathrm{in}{L}^r\left(\mathsf{\Omega }\right)\mathrm{and}{L}^{\gamma }(\partial \mathsf{\Omega }),$$

for $1\le \gamma <min\{2_{\beta ,\alpha }^{*},2_{b,\alpha }^{*}\}$, $1\le r<2_{\alpha }^{*}$, $0<b<2\alpha$, and $0\le \beta <\alpha \gamma +N(1-{\displaystyle \frac{\gamma }{2}})$. We have to establish that ${\phi }_n\to {\phi }_2$ in X. Arguing by contradiction, suppose that $\parallel {\phi }_2\parallel <lim{inf}_{n\to \infty }\parallel {\phi }_n\parallel$. Hence, we deduce that

$$\begin{array}{cc}\hfill \langle {\mathcal{J}}_{\lambda }^{\prime }\left({\phi }_2\right),{\phi }_2\rangle & =\parallel {\phi }_2{\parallel }^2-P\left({\phi }_2\right)-\lambda K\left({\phi }_2\right)\hfill \\ \hfill & <lim\underset{n\to \infty }{inf}\left(\parallel {\phi }_n{\parallel }^2-P\left({\phi }_n\right)-\lambda K\left({\phi }_n\right)\right)\hfill \\ \hfill & =0.\hfill \end{array}$$

Comparing this inequality with ${\phi }_2\in {\mathcal{Q}}_{\lambda }^{-}$, we have a contradiction. This way, we conclude that ${\phi }_n\to {\phi }_2$ in X, as $n\to \infty$, and hence ${\mathcal{J}}_{\lambda }\left({\phi }_2\right)={\beta }_0^{-}$. Similar to the proof of Lemma 9, we note that ${\mathcal{J}}_{\lambda }\left({\phi }_2\right)={\mathcal{J}}_{\lambda }\left(\right|{\phi }_2\left|\right)$, and $|{\phi }_2|\in {\mathcal{Q}}_{\lambda }^{-}$ is a solution to problem (1). From Lemma 4, we may suppose that ${\phi }_2$ is a non-negative solution to problem (1). Then, due to the Harnack inequality (see Zhang-Liu [28]), we conclude that ${\phi }_2>0$. □

Proof of Theorem 1. 

Utilizing Lemmas 9 and 10, we have two positive solutions ${\phi }_1$ and ${\phi }_2$, such that ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$ and ${\phi }_2\in {\mathcal{Q}}_{\lambda }^{-}$, respectively. Moreover, by Lemma 5, we know that ${\mathcal{Q}}_{\lambda }^{+}\cap {\mathcal{Q}}_{\lambda }^{-}=\varnothing$. It follows that ${\phi }_1$ and ${\phi }_2$ are exactly two distinct positive solutions of problem (1). □

5. Proof of Theorem 2

In this section, using again the Nehari method, we establish our second result, namely Theorem 2, which says that for sufficiently small $\lambda >0$, problem (1) admits at least two positive solutions in the critical case $q=2_{\beta ,\alpha }^{*}$. More precisely, our proof will be divided into two lemmas (Lemmas 14 and 17), but first, we need some auxiliary results. Here, we suppose $(2N-\mu )/N\le p\le 2_{\mu ,\alpha }^{*}$, and set $f\left(x\right)\equiv 1$. For simplicity, we also use the following notation:

$${\overline{\lambda }}^{*}:=\overline{\lambda }\left(2_{\beta ,\alpha }^{*}\right),{\mathsf{\Xi }}_{q^{*}}:=\mathsf{\Xi }\left(2_{\beta ,\alpha }^{*}\right),\mathrm{and}\mathcal{J}\left(\phi \right):={\mathcal{J}}_{\lambda ,q:=2_{\beta ,\alpha }^{*}}\left(\phi \right).$$

Inspired by Ghoussoub et al. [29], we state the following results.

Lemma 11.

([29]). If $0\le \beta <2\alpha <2$, $N>2\alpha$ and $0\le \kappa <\kappa \left(\alpha \right)$, then any positive extremal $\phi \in X$ for ${\mathsf{\Xi }}_{q^{*}}$ satisfies $\phi \in C^1({\mathbb{R}}^n\backslash \left\{0\right\})$ and

$$\underset{x\to 0}{lim}{\left|x\right|}^{{\nu }_{-}\left(\kappa \right)}\phi \left(x\right)={\overline{\mathsf{\Lambda }}}_0\mathrm{and}\underset{x\to \infty }{lim}{\left|x\right|}^{{\nu }_{+}\left(\kappa \right)}\phi \left(x\right)={\mathsf{\Lambda }}_{\infty },$$

(30)

where ${\overline{\mathsf{\Lambda }}}_0,{\mathsf{\Lambda }}_{\infty }>0$ and ${\nu }_{-}\left(\kappa \right)$(resp., ${\nu }_{+}\left(\kappa \right)$) is the unique solution in $(0,{\displaystyle \frac{0,N-2\alpha }{2}})$ (resp., in $({\displaystyle \frac{N-2\alpha }{2}},N-\alpha )$) of the equation

$${\mathsf{\Psi }}_{N,\alpha }\left(t\right):=2^{2\alpha }{\displaystyle \frac{\mathsf{\Gamma }\left(\frac{t+2\alpha }{2}\right)\mathsf{\Gamma }\left(\frac{N-t}{2}\right)}{\mathsf{\Gamma }\left(\frac{N-t-2\alpha }{2}\right)\mathsf{\Gamma }\left(\frac{t}{2}\right)}}=\kappa ,$$

with ${\nu }_{-}\left(0\right)=0$ and ${\nu }_{+}\left(0\right)=N-2\alpha$. Further, we can find positive constants $C_2,C_3>0$ such that

$${\displaystyle \frac{C_2}{{\left|x\right|}^{{\nu }_{-}\left(\kappa \right)}+{\left|x\right|}^{{\nu }_{+}\left(\kappa \right)}}}\le \phi \left(x\right)\le {\displaystyle \frac{C_3}{{\left|x\right|}^{{\nu }_{-}\left(\kappa \right)}+{\left|x\right|}^{{\nu }_{+}\left(\kappa \right)}}}\mathrm{for}\mathrm{all}x\in {\mathbb{R}}^N\backslash \left\{0\right\}.$$

Let ${\phi }^{*}\left(x\right)$ be a positive weak solution of (1), and define ${\phi }_{\epsilon }\left(x\right)={\epsilon }^{{\displaystyle \frac{2\alpha -N}{2}}}{\phi }^{*}\left({\displaystyle \frac{x}{\epsilon }}\right)$ with $\epsilon >0$ in ${\mathbb{R}}^N$. Clearly, ${\phi }_{\epsilon }\left(x\right)$ is also a solution of (1). Take $\rho >0$ small enough such that $B_{2\rho }\left(0\right)\subset \overline{\mathsf{\Omega }}$, $B_{2\rho }\left(0\right)=\{x\in {\mathbb{R}}^N:\left|x\right|<2\rho \}$. Choose the radial cut-off function $\xi \left(x\right)\in C_0^{\infty }\left(\overline{\mathsf{\Omega }}\right)$ such that $0\le \xi \left(x\right)\le 1$ in $B_{2\rho }\left(0\right)$, $\xi \left(x\right)=1$ in $B_{\rho }\left(0\right)$, and $\xi \left(x\right)=0$ in $B_{2\rho }^c\left(0\right)$. One can check that $\xi \left(x\right){\phi }_{\epsilon }\left(x\right)$ belongs in X. For any $\epsilon >0$, we set

$$U_{\epsilon }\left(x\right)=\xi \left(x\right){\phi }_{\epsilon }\left(x\right)\mathrm{for}x\in {\mathbb{R}}^N,$$

(31)

and have the following Lemmas.

Lemma 12.

([29]). If $U_{\epsilon }$ is given by (31), and ${\phi }_1$ is a positive solution of (1), then for all $\epsilon >0$ small enough, we get

(i) 

$\parallel U_{\epsilon }{\parallel }^2\le {\parallel {\phi }_{\epsilon }\parallel }^2+O\left({\epsilon }^{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}\right)$;

(ii) 

${\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s={\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left({\epsilon }^{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}\right)$.

Lemma 13.

If $U_{\epsilon }$ is given by (31), and ${\phi }_1$ is a positive solution of (1), then for all $\epsilon >0$ small enough we get

$$\begin{array}{cc}\hfill {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1+t{U}_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s=& {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|t{U}_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+2_{\beta ,\alpha }^{*}t{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1{|}^{2_{\beta ,\alpha }^{*}-2}}{{\left|x\right|}^{\beta }}}{U}_{\epsilon }{\phi }_1\mathrm{d}s\hfill \\ \hfill & +2_{\beta ,\alpha }^{*}{t}^{2_{\beta ,\alpha }^{*}-1}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}-2}}{{\left|x\right|}^{\beta }}}{U}_{\epsilon }{\phi }_1\mathrm{d}s+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right).\hfill \end{array}$$

(32)

We note that (32) reflects Equation (17) in Brezis and Nirenberg [30] (see the proof of Theorem 1 (p. 133) and use Lemma 4) with only minor modifications. Therefore, we omit the proof of Lemma 13 here. Next, we give the existence result of a positive solution to problem (1) on ${\mathcal{Q}}_{\lambda }^{+}$.

Lemma 14.

If $0<\lambda <{\overline{\lambda }}^{*}$, then the functional $\mathcal{J}$ admits a minimizer ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$ satisfying the following conditions:

(i) 

$\mathcal{J}\left({\phi }_1\right)={\beta }_0={\beta }_0^{+}<0$;

(ii) 

${\phi }_1$ is a positive solution to problem (1).

The proof of Lemma 14 repeats the proof of previous Lemma 9 for the functional ${\mathcal{J}}_{\lambda }$ with $q=2_{\beta ,\alpha }^{*}$. In obtaining the existence result on ${\mathcal{Q}}_{\lambda }^{-}$, the following lemmas play a crucial role; hence, we have to properly manipulate the ${\left(PS\right)}_{{\beta }_0^{-}}$ condition.

Lemma 15.

If ${\phi }_1$ is the local minimum in Lemma 14, then for $\epsilon >0$ small enough, we obtain

$$\underset{t\ge 0}{sup}\mathcal{J}({\phi }_1+t{U}_{\epsilon })<{\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}.$$

Proof. 

Consider the functional

$$\begin{array}{c}\hfill \mathcal{J}({\phi }_1+t{U}_{\epsilon })={\displaystyle \frac{1}{2}}{\parallel {\phi }_1+t{U}_{\epsilon }\parallel }^2-{\displaystyle \frac{1}{2p}}P({\phi }_1+t{U}_{\epsilon })-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1+t{U}_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s.\end{array}$$

We know that

$$\parallel {\phi }_1+t{U}_{\epsilon }{\parallel }^2=\parallel {\phi }_1{\parallel }^2+t^2{\parallel U_{\epsilon }\parallel }^2+2t\langle {\phi }_1,U_{\epsilon }\rangle -2t\kappa {\int }_{\mathsf{\Omega }}{\displaystyle \frac{{\phi }_1{U}_{\epsilon }}{{\left|x\right|}^{2\alpha }}}\mathrm{d}x,$$

and so we get

$$\begin{array}{cc}\hfill \mathcal{J}({\phi }_1+t{U}_{\epsilon })=& {\displaystyle \frac{1}{2}}\parallel {\phi }_1{\parallel }^2+{\displaystyle \frac{t^2}{2}}{\parallel U_{\epsilon }\parallel }^2+t\langle {\phi }_1,U_{\epsilon }\rangle -t\kappa {\int }_{\mathsf{\Omega }}{\displaystyle \frac{{\phi }_1{U}_{\epsilon }}{{\left|x\right|}^{2\alpha }}}\mathrm{d}x\hfill \\ \hfill & -{\displaystyle \frac{1}{2p}}P({\phi }_1+t{U}_{\epsilon })-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1+t{U}_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s.\hfill \end{array}$$

Notice that ${\phi }_1$ is a minimizer for $\mathcal{J}$, then one has

$${\displaystyle \frac{1}{2}}{\parallel {\phi }_1\parallel }^2=\mathcal{J}\left({\phi }_1\right)+{\displaystyle \frac{1}{2p}}P\left({\phi }_1\right)+{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s.$$

(33)

Substituting the test function $\xi {\phi }_1$ into ${\mathcal{J}}^{\prime }\left(\phi \right)=0$ in X yields

$$\begin{array}{cc}\hfill & t\langle {\phi }_1,U_{\epsilon }\rangle -t\kappa {\int }_{\mathsf{\Omega }}{\displaystyle \frac{{\phi }_1{U}_{\epsilon }}{{\left|x\right|}^{2\alpha }}}\mathrm{d}x\hfill \\ \hfill =& t{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_1{\left(y\right)|}^p{\left|{\phi }_1\left(x\right)\right|}^{p-2}{\phi }_1\left(x\right)U_{\epsilon }}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y+t\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1{|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{U}_{\epsilon }\mathrm{d}s.\hfill \end{array}$$

(34)

Equations (32)–(34) give us

$$\begin{array}{cc}\hfill \mathcal{J}({\phi }_1+t{U}_{\epsilon })=& \mathcal{J}\left({\phi }_1\right)+{\displaystyle \frac{1}{2p}}P\left({\phi }_1\right)+t{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_1{\left(y\right)|}^p{\left|{\phi }_1\left(x\right)\right|}^{p-2}{\phi }_1\left(x\right)U_{\epsilon }}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & -{\displaystyle \frac{1}{2p}}P({\phi }_1+t{U}_{\epsilon })+{\displaystyle \frac{t^2}{2}}{\parallel U_{\epsilon }\parallel }^2-{\displaystyle \frac{\lambda t^{2_{\beta ,\alpha }^{*}}}{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\hfill \\ \hfill & -\lambda t^{2_{\beta ,\alpha }^{*}-1}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\phi }_1\mathrm{d}s+o({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}).\hfill \end{array}$$

From Ghoussoub et al. [29] and Abdellaoui et al. [31], one can find a positive constant $C_4>0$ such that

$$C_4^{-1}\le {\left|x\right|}^{{\nu }_{-}\left(\kappa \right)}{\phi }_1\left(x\right)\le C_4\mathrm{for}\mathrm{all}x\in \mathsf{\Omega }.$$

Then, there exists $l>0$ such that

$$\underset{x\to 0}{lim}{\left|x\right|}^{{\nu }_{-}\left(\kappa \right)}{\phi }_1\left(x\right)=l.$$

Hence, we have

$$\begin{array}{cc}\hfill & {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\phi }_1\mathrm{d}s\le C_4{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\left|x\right|}^{-{\nu }_{-}\left(\kappa \right)}\mathrm{d}s\hfill \\ \hfill =& C_4{\int }_{B_{{\delta }_0}}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\left|x\right|}^{-{\nu }_{-}\left(\kappa \right)}\mathrm{d}s+C_4{\int }_{\overline{\mathsf{\Omega }}\backslash B_{{\delta }_0}}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\left|x\right|}^{-{\nu }_{-}\left(\kappa \right)}\mathrm{d}s\hfill \\ \hfill =& C_4{\epsilon }^{N+{\displaystyle \frac{2\alpha -N}{2}}l-\beta -{\nu }_{-}\left(\kappa \right)}{\int }_{B_{{\epsilon }^{-1}\delta }}{\displaystyle \frac{|{\phi }^{*}{\left(x\right)|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\left|x\right|}^{-{\nu }_{-}\left(\kappa \right)}\mathrm{d}x+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right)\hfill \\ \hfill =& C_4{\epsilon }^{N+{\displaystyle \frac{2\alpha -N}{2}}l-\beta -{\nu }_{-}\left(\kappa \right)}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{|{\phi }^{*}{\left(x\right)|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\left|x\right|}^{-{\nu }_{-}\left(\kappa \right)}\mathrm{d}x+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right)\hfill \\ \hfill =& C_4{\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{|{\phi }^{*}{\left(x\right)|}^{2_{\beta ,\alpha }^{*}-1}}{{\left|x\right|}^{\beta }}}{\left|x\right|}^{-{\nu }_{-}\left(\kappa \right)}\mathrm{d}x+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right)\hfill \\ \hfill =& C_5{\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right),\hfill \end{array}$$

for some $C_5>0$$(\epsilon \to 0)$. In addition, we obtain

$$\begin{array}{cc}\hfill & {\displaystyle \frac{1}{2p}}P\left({\phi }_1\right)+t{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_1{\left(y\right)|}^p{\left|{\phi }_1\left(x\right)\right|}^{p-2}{\phi }_1\left(x\right)U_{\epsilon }}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y-{\displaystyle \frac{1}{2p}}P({\phi }_1+t{U}_{\epsilon })\hfill \\ \hfill =& {\displaystyle \frac{1}{2p}}{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_1{\left(x\right)|}^p{\left|{\phi }_1\left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y+t{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_1{\left(y\right)|}^p{\left|{\phi }_1\left(x\right)\right|}^{p-2}{\phi }_1\left(x\right)U_{\epsilon }}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & -{\displaystyle \frac{1}{2p}}{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_1\left(x\right)+t{U}_{\epsilon }{|}^p{|{\phi }_1\left(y\right)+t{U}_{\epsilon }|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill =& -{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\int }_0^{t{U}_{\epsilon }}\left(|{\phi }_1{\left(y\right)+\tau |}^p|{\phi }_1{\left(x\right)+\tau |}^{p-1}-|{\phi }_1{\left(y\right)|}^p{\left|{\phi }_1\left(x\right)\right|}^{p-1}\right)\mathrm{d}\tau }{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill \le & 0.\hfill \end{array}$$

Now, we deduce by Lemma 11 that the last integral is finite by the asymptotics (30). We can find $\tilde{c}>0$ such that

$$\mathcal{J}({\phi }_1+t{U}_{\epsilon })\le \mathcal{J}\left({\phi }_1\right)+{\displaystyle \frac{t^2}{2}}{\parallel {\phi }_{\epsilon }\parallel }^2-\lambda {\displaystyle \frac{t^{2_{\beta ,\alpha }^{*}}}{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s-\tilde{c}{\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right).$$

Setting

$$\mathcal{H}\left(t\right)={\displaystyle \frac{t^2}{2}}{\parallel {\phi }_{\epsilon }\parallel }^2-\lambda {\displaystyle \frac{t^{2_{\beta ,\alpha }^{*}}}{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\mathrm{for}t>0,$$

it is easy to obtain that $\mathcal{H}$ achieves its maximum at ${\tilde{t}}_0={\left({\displaystyle \frac{\parallel {\phi }_{\epsilon }{\parallel }^2}{\lambda {\int }_{\partial \mathsf{\Omega }}\frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}\mathrm{d}s}}\right)}^{{\displaystyle \frac{1}{2_{\beta ,\alpha }^{*}-2}}}$, and

$$\underset{t\to \infty }{lim}\mathcal{H}\left(t\right)=-\infty ,\mathcal{H}\left({\tilde{t}}_0\right)=\left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2_{\beta ,\alpha }^{*}}}\right){\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\parallel {\phi }_{\epsilon }\parallel }^{{\displaystyle \frac{{22}_{\beta ,\alpha }^{*}}{2_{\beta ,\alpha }^{*}-2}}}{\left({\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\right)}^{-{\displaystyle \frac{2}{2_{\beta ,\alpha }^{*}-2}}},$$

which leads to the inequality

$$\mathcal{H}\left(t\right)\le \left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2_{\beta ,\alpha }^{*}}}\right){\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\parallel {\phi }_{\epsilon }\parallel }^{{\displaystyle \frac{{22}_{\beta ,\alpha }^{*}}{2_{\beta ,\alpha }^{*}-2}}}{\left({\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\right)}^{-{\displaystyle \frac{2}{2_{\beta ,\alpha }^{*}-2}}}.$$

Considering the fact that ${\phi }_{\epsilon }$ is an extremal for (9) and (8), we have

$$\parallel {\phi }_{\epsilon }{\parallel }^2={\mathsf{\Xi }}_{q^{*}}{\left({\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\right)}^{{\displaystyle \frac{2}{2_{\beta ,\alpha }^{*}}}},$$

which implies

$${\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{2_{\beta ,\alpha }^{*}}{2_{\beta ,\alpha }^{*}-2}}}={\parallel {\phi }_{\epsilon }\parallel }^{{\displaystyle \frac{{22}_{\beta ,\alpha }^{*}}{2_{\beta ,\alpha }^{*}-2}}}{\left({\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\right)}^{-{\displaystyle \frac{2}{2_{\beta ,\alpha }^{*}-2}}}.$$

Since ${\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2_{\beta ,\alpha }^{*}}}={\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}$ and ${\displaystyle \frac{2_{\beta ,\alpha }^{*}}{2_{\beta ,\alpha }^{*}-2}}={\displaystyle \frac{N-\beta }{2\alpha -\beta }}$, we derive that

$$\mathcal{H}\left(t\right)\le {\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}\mathrm{for}\mathrm{all}t>0.$$

Thus, we conclude that

$$\begin{array}{cc}\hfill \mathcal{J}({\phi }_1+t{U}_{\epsilon })& \le \mathcal{J}\left({\phi }_1\right)+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}-\tilde{c}{\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}+o\left({\epsilon }^{{\displaystyle \frac{{\nu }_{+}\left(\kappa \right)-{\nu }_{-}\left(\kappa \right)}{2}}}\right)\hfill \\ \hfill & <{\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}\mathrm{for}\mathrm{all}t>0.\hfill \end{array}$$

□

Lemma 16.

Assume there is a minimizing sequence $\left\{{\phi }_n\right\}$ for $\mathcal{J}$ on ${\mathcal{Q}}_{\lambda }^{+}$ satisfying the following:

(i) 

$\mathcal{J}\left({\phi }_n\right)=\varrho +o\left(1\right)$ with $\varrho <{\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}$;

(ii) 

${\mathcal{J}}^{\prime }\left({\phi }_n\right)=o\left(1\right)$ in X.

Then, there exists a subsequence of $\left\{{\phi }_n\right\}$, which is strongly convergent in X.

Proof. 

We deduce by Lemma 6 that there exists a subsequence $\left\{{\phi }_n\right\}$ and $\phi$ such that

$$\begin{array}{cc}\hfill & {\phi }_n\rightharpoonup \phi \mathrm{weakly}\mathrm{in}X;\hfill \\ \hfill & {\phi }_n\to \phi \mathrm{strongly}\mathrm{in}{L}^r\left(\mathsf{\Omega }\right)\mathrm{for}r\in [1,2_{\alpha }^{*}).\hfill \end{array}$$

(35)

Furthermore, assumption (ii) gives

$$\langle {\mathcal{J}}^{\prime }\left(\phi \right),u\rangle =0\mathrm{for}\mathrm{any}u\in X.$$

Thus, $\phi$ is a solution in X for problem (1) with $\mathcal{J}\left(\phi \right)\ge {\beta }_0$.

Now, we prove $\phi \ne 0$. We argue by contradiction; hence, we assume $\phi \equiv 0$. Using (35) and $(2N-\mu )/N\le p<2_{\alpha }^{*}$, we have

$${\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_n{\left(x\right)|}^p{\left|{\phi }_n\left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\to {\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{{\left|\phi \left(x\right)\right|}^p{\left|\phi \left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y=0,$$

which shows

$${\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_n{\left(x\right)|}^p{\left|{\phi }_n\left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y=o\left(1\right)(n\to \infty ).$$

This fact and assumption (ii) yield

$$\begin{array}{cc}\hfill \parallel {\phi }_n{\parallel }^2=& {\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_n{\left(x\right)|}^p{\left|{\phi }_n\left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y+\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\hfill \\ \hfill =& \lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)(n\to \infty ),\hfill \end{array}$$

(36)

which, together with assumption (i) shows that

$$\begin{array}{cc}\hfill \mathcal{J}\left({\phi }_n\right)& ={\displaystyle \frac{1}{2}}{\parallel {\phi }_n\parallel }^2-{\displaystyle \frac{1}{2p}}{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\phi }_n{\left(x\right)|}^p{\left|{\phi }_n\left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\hfill \\ \hfill & =\left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2_{\beta ,\alpha }^{*}}}\right)\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)\hfill \\ \hfill & ={\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)\hfill \\ \hfill & =\varrho +o\left(1\right)(n\to \infty ).\hfill \end{array}$$

It follows from $\varrho <{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}$ that

$${\lambda }^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s<{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}+o\left(1\right)(n\to \infty ).$$

(37)

Again, (8) and (36) give

$${\lambda }^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\ge {\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}+o\left(1\right),$$

which contradicts (37). Hence, $\phi \ne 0$ with $\mathcal{J}\ge {\beta }_0$.

Set ${\overline{\zeta }}_n={\phi }_n-\phi$. Inspired by Ghoussoub and Yua [32], we deduce by the Brezis–Lieb Lemma (see also [33]) that

$${\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s={\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{{\left|\phi \right|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)(n\to \infty ).$$

Consequently, due to weak convergence of ${\overline{\zeta }}_n\rightharpoonup 0$ in X, for n large enough, we have

$$\begin{array}{cc}\hfill {\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}& >\mathcal{J}(\phi +{\overline{\zeta }}_n)\hfill \\ \hfill & =\mathcal{J}\left(\phi \right)+{\displaystyle \frac{1}{2}}{\parallel {\overline{\zeta }}_n\parallel }^2-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)\hfill \\ \hfill & \ge {\beta }_0+{\displaystyle \frac{1}{2}}{\parallel {\overline{\zeta }}_n\parallel }^2-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right),\hfill \end{array}$$

which leads to

$${\displaystyle \frac{1}{2}}{\parallel {\overline{\zeta }}_n\parallel }^2-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s<{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}+o\left(1\right).$$

(38)

Noting that $\left\{{\overline{\zeta }}_n\right\}$ is uniformly bounded by assumption (ii), and $\phi$ is a solution of (1), one has

$$\begin{array}{cc}\hfill o\left(1\right)& =\langle {\mathcal{J}}^{\prime }\left({\overline{\zeta }}_n\right),{\overline{\zeta }}_n\rangle \hfill \\ \hfill & ={\mathcal{J}}^{\prime }\left(\phi \right)+{\parallel {\overline{\zeta }}_n\parallel }^2-\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)\hfill \\ \hfill & =\parallel {\overline{\zeta }}_n{\parallel }^2-\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right),\hfill \end{array}$$

for ${\mathcal{J}}^{\prime }\left(\phi \right)=0$. It follows that

$$\parallel {\overline{\zeta }}_n{\parallel }^2-\lambda {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s=o\left(1\right).$$

(39)

If (38) and (39) hold, then $\left\{{\overline{\zeta }}_n\right\}$ admits a subsequence, which converges strongly to zero. We again argue by contradiction. Suppose that $\left\{{\overline{\zeta }}_n\right\}$ is bounded away from zero, that is, there exists a constant ${\tilde{c}}_0$ such that $\parallel {\overline{\zeta }}_n\parallel >{\tilde{c}}_0>0$. Using (38) and (39), we deduce that

$$\begin{array}{cc}\hfill {\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}& \le {\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\parallel {\overline{\zeta }}_n\parallel }^2+o\left(1\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}{\parallel {\overline{\zeta }}_n\parallel }^2-{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\overline{\zeta }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s+o\left(1\right)\hfill \\ \hfill & <{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}+o\left(1\right),\hfill \end{array}$$

which is a contradiction. Thus, up to a subsequence, ${\overline{\zeta }}_n\to 0$ strongly in X, which shows that ${\phi }_n\to \phi$ strongly in X, too. □

Next, we prove the existence results for problem (1) on the submanifold ${\mathcal{Q}}_{\lambda }^{-}$.

Lemma 17.

If $0<\lambda <{\overline{\lambda }}^{*}$, then the functional $\mathcal{J}$ admits a minimizer ${\phi }_2\in {\mathcal{Q}}_{\lambda }^{-}$ satisfying the following conditions:

(i) 

$\mathcal{J}\left({\phi }_2\right)={inf}_{\phi \in {\mathcal{Q}}^{-}}\mathcal{J}\left(\phi \right)={\beta }_0^{-}<{\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}$;

(ii) 

${\phi }_2$ is a nontrivial non-negative solution to problem (1).

Proof. 

In what follows, in order to prove

$${\beta }_0^{-}<{\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}},$$

(40)

by Lemma 8, there exists a unique $t^{-}\left(\phi \right)>0$ such that $t^{-}\left(\phi \right)\phi \in {\mathcal{Q}}_{\lambda }^{-}$. Set

$$\begin{array}{cc}\hfill & {\mathcal{Y}}_1:=\left\{\phi \in X\backslash \left\{0\right\}:t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)>\parallel \phi \parallel \right\}\cup \left\{0\right\},\hfill \\ \hfill & {\mathcal{Y}}_2:=\left\{\phi \in X\backslash \left\{0\right\}:t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)<\parallel \phi \parallel \right\}.\hfill \end{array}$$

Therefore, ${\mathcal{Q}}_{\lambda }^{-}$ disconnects X into two connected components ${\mathcal{Y}}_1$ and ${\mathcal{Y}}_2$, and $X\backslash {\mathcal{Q}}_{\lambda }^{-}={\mathcal{Y}}_1\cup {\mathcal{Y}}_2$. For $\phi \in {\mathcal{Q}}_{\lambda }^{+}$, there exist unique $t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)>0$ and $t^{+}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)>0$ such that $t^{+}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)<t_0<t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)$, $t^{+}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)\in {\mathcal{Q}}_{\lambda }^{+}$, and $t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)\in {\mathcal{Q}}_{\lambda }^{-}$. Since $\phi \in {\mathcal{Q}}_{\lambda }^{+}$, we obtain $t^{+}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right){\displaystyle \frac{1}{\parallel \phi \parallel }}=1$. Using $t^{+}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)<t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)$, we have $t^{-}\left({\displaystyle \frac{\phi }{\parallel \phi \parallel }}\right)>\parallel \phi \parallel$ and ${\mathcal{Q}}_{\lambda }^{+}\subset {\mathcal{Y}}_1$. More precisely, ${\phi }_1\in {\mathcal{Y}}_1$.

Next, we prove that there exists ${\upsilon }_0>0$ such that ${\phi }_1+{\upsilon }_0{U}_{\epsilon }\in {\mathcal{Y}}_2$. Thus, for all ${\upsilon }_0>0$, there exists a positive constant $\overline{C}$ such that

$$0<t^{-}\left({\displaystyle \frac{{\phi }_1+{\upsilon }_0{U}_{\epsilon }}{\parallel {\phi }_1+{\upsilon }_0{U}_{\epsilon }\parallel }}\right)<\overline{C}.$$

(41)

We argue by contradiction, so we assume that there exists a subsequence $\left\{{\upsilon }_n\right\}$ such that

$${\upsilon }_n\to \infty ,\mathrm{and}{t}^{-}\left({\displaystyle \frac{{\phi }_1+{\upsilon }_n{U}_{\epsilon }}{\parallel {\phi }_1+{\upsilon }_n{U}_{\epsilon }\parallel }}\right)\to \infty ,asn\to \infty .$$

Set ${\tilde{\varphi }}_n={\displaystyle \frac{{\phi }_1+{\upsilon }_n{U}_{\epsilon }}{\parallel {\phi }_1+{\upsilon }_n{U}_{\epsilon }\parallel }}$. Lemma 8 yields that $t^{-}\left({\tilde{\varphi }}_n\right){\tilde{\varphi }}_n\in {\mathcal{Q}}_{\lambda }^{-}\subset {\mathcal{Q}}_{\lambda }$. Applying the Lebesgue dominated convergence theorem, we obtain

$$\begin{array}{cc}\hfill {\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s=& {\displaystyle \frac{1}{\parallel {\phi }_1+{\upsilon }_n{U}_{\epsilon }{\parallel }^{2_{\beta ,\alpha }^{*}}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_1+{\upsilon }_n{U}_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\hfill \\ \hfill =& {\displaystyle \frac{1}{\parallel \frac{{\phi }_1}{{\upsilon }_n}+U_{\epsilon }{\parallel }^{2_{\beta ,\alpha }^{*}}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|\frac{{\phi }_1}{{\upsilon }_n}+U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\hfill \end{array}$$

for ${\upsilon }_n\to \infty$, as $n\to \infty$. Thus, we also have

$${\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\phi }_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\to {\displaystyle \frac{1}{\parallel U_{\epsilon }{\parallel }^{2_{\beta ,\alpha }^{*}}}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|U_{\epsilon }{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}sasn\to \infty .$$

It follows that

$$\begin{array}{cc}\hfill \mathcal{J}\left(t^{-}\left({\tilde{\varphi }}_n\right){\tilde{\varphi }}_n\right)=& {\displaystyle \frac{1}{2}}{\left[t^{-}\left({\tilde{\varphi }}_n\right)\right]}^2{\parallel {\tilde{\varphi }}_n\parallel }^2-{\displaystyle \frac{1}{2p}}{\left[t^{-}\left({\tilde{\varphi }}_n\right)\right]}^{2p}{\int }_{\mathsf{\Omega }}{\int }_{\mathsf{\Omega }}h\left(x\right){\displaystyle \frac{|{\tilde{\varphi }}_n{|}^p\left(x\right){\left|{\tilde{\varphi }}_n\left(y\right)\right|}^p}{{|x-y|}^{\mu }}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & -{\displaystyle \frac{\lambda }{2_{\beta ,\alpha }^{*}}}{\left[t^{-}\left({\tilde{\varphi }}_n\right)\right]}^{2_{\beta ,\alpha }^{*}}{\int }_{\partial \mathsf{\Omega }}{\displaystyle \frac{|{\tilde{\varphi }}_n{|}^{2_{\beta ,\alpha }^{*}}}{{\left|x\right|}^{\beta }}}\mathrm{d}s\to -\infty ,\hfill \end{array}$$

which contradicts the fact that $\mathcal{J}$ is bounded below. Hence, (41) holds.

Let ${\upsilon }_0:={\displaystyle \frac{|{\overline{C}}^2-\parallel {\phi }_1{{\parallel }^2|}^{\frac{1}{2}}}{\parallel U_{\epsilon }\parallel }}+1$. Then, we get

$$\begin{array}{cc}\hfill \parallel {\phi }_1+{\upsilon }_0{U}_{\epsilon }{\parallel }^2& =\parallel {\phi }_1{\parallel }^2+{\upsilon }_0^2{\parallel U_{\epsilon }\parallel }^2+2{\upsilon }_0\left(C(N,\alpha )\langle {\phi }_1,U_{\epsilon }\rangle -\kappa {\int }_{\mathsf{\Omega }}{\displaystyle \frac{{\phi }_1{U}_{\epsilon }}{{\left|x\right|}^{2\alpha }}}\mathrm{d}x\right)\hfill \\ \hfill & \ge \parallel {\phi }_1{\parallel }^2+|{\overline{C}}^2-\parallel {\phi }_1{\parallel }^2|\hfill \\ \hfill & \ge {\overline{C}}^2\hfill \\ \hfill & >|t^{-}\left({\displaystyle \frac{{\phi }_1+{\upsilon }_0{U}_{\epsilon }}{\parallel {\phi }_1+{\upsilon }_0{U}_{\epsilon }\parallel }}\right){|}^2,\hfill \end{array}$$

which implies that

$$t^{-}\left({\displaystyle \frac{{\phi }_1+{\upsilon }_0{U}_{\epsilon }}{\parallel {\phi }_1+{\upsilon }_0{U}_{\epsilon }\parallel }}\right)<\parallel {\phi }_1+{\upsilon }_0{U}_{\epsilon }\parallel ,$$

and so ${\phi }_1+{\upsilon }_0{U}_{\epsilon }\in {\mathcal{Y}}_2$.

We now introduce the following notation

$$\begin{array}{cc}\hfill \tilde{\mathsf{\Gamma }}:=& \{\eta \in (C[0,1],X):\eta \left(0\right)={\phi }_1\mathrm{and}\eta \left(1\right)={\phi }_1+{\upsilon }_0{U}_{\epsilon }\},\hfill \\ \hfill \overline{\vartheta }:=& \underset{\eta \in \tilde{\mathsf{\Gamma }}}{inf}\underset{\varsigma \in [0,1]}{max}\mathcal{J}\left(\eta \left(\varsigma \right)\right),\overline{\xi }\left(\varsigma \right)={\phi }_1+\varsigma {\upsilon }_0{U}_{\epsilon },\mathrm{for}\varsigma \in [0,1].\hfill \end{array}$$

It follows that $\overline{\xi }\left(0\right)\in {\mathcal{Y}}_1$ and $\overline{\xi }\left(1\right)\in {\mathcal{Y}}_2$. Hence, there exists ${\varsigma }_0\in (0,1)$ such that $\overline{\xi }\left({\varsigma }_0\right)\in {\mathcal{Q}}_{\lambda }^{-}$ and $\overline{\vartheta }\ge {\beta }_0^{-}$. Lemma 16 gives

$${\beta }_0^{-}<\overline{\vartheta }<{\beta }_0+{\displaystyle \frac{2\alpha -\beta }{2(N-\beta )}}{\lambda }^{{\displaystyle \frac{2\alpha -N}{2\alpha -\beta }}}{\mathsf{\Xi }}_{q^{*}}^{{\displaystyle \frac{N-\beta }{2\alpha -\beta }}}.$$

From the Ekeland’s variational principle, there exists a sequence $\left\{{\phi }_n\right\}\subset {\mathcal{Q}}_{\lambda }^{-}$ such that

$$\mathcal{J}\left({\phi }_n\right)={\beta }_0^{-}+o\left(1\right)\mathrm{and}{\mathcal{J}}^{\prime }\left({\phi }_n\right)=o\left(1\right)\mathrm{in}{X}^{*}.$$

Again, by Lemma 16 and (40), there exist a relabeled subsequence $\left\{{\phi }_n\right\}$ and ${\phi }_2$ such that ${\phi }_n\to {\phi }_2$ strongly in X. Hence, ${\phi }_2\in {\mathcal{Q}}_{\lambda }^{-}$ and $\mathcal{J}\left({\phi }_n\right)\to \mathcal{J}\left({\phi }_2\right)={\beta }_0^{-}$ as $n\to \infty$.

Considering the fact that $\mathcal{J}\left({\phi }_2\right)=\mathcal{J}\left(\right|{\phi }_2\left|\right)$, and $|{\phi }_2|\in {\mathcal{Q}}_{\lambda }^{-}$ is a solution of (1), we may suppose that ${\phi }_2$ is a non-negative solution to problem (1). Furthermore, by the maximum principle (see Silvestre [34]), we obtain ${\phi }_2>0$ in X. This concludes the proof. □

Proof of Theorem 2. 

Combining Lemma 14 and Lemma 17, we already have two positive solutions ${\phi }_1$ and ${\phi }_2$ such that ${\phi }_1\in {\mathcal{Q}}_{\lambda }^{+}$ and ${\phi }_2\in {\mathcal{Q}}_{\lambda }^{-}$, respectively. Now, by Lemma 5, we know that ${\mathcal{Q}}_{\lambda }^{+}\cap {\mathcal{Q}}_{\lambda }^{-}=\varnothing$. It follows that ${\phi }_1$ and ${\phi }_2$ are exactly two distinct positive solutions of problem (1). □