Joint Transmit and Receive Beamforming Design for DPC-Based MIMO DFRC Systems

Abstract

This paper proposes an optimal beamforming strategy for a downlink multi-user multi-input–multi-output (MIMO) dual-function radar communication (DFRC) system with dirty paper coding (DPC) adopted at the transmitter. We aim to achieve the maximum weighted sum rate of communicating users while adhering to a predetermined transmit covariance constraint for radar performance assurance. To make the intended problem trackable, we leverage the equivalence of the weighted sum rate and the weighted minimum mean squared error (MMSE) to reframe the issue and devise a block coordinate descent (BCD) approach to iteratively calculate transmit and receive beamforming solutions. Through this methodology, we demonstrate that the optimal receive beamforming aligns with the traditional MMSE approach, whereas the optimal transmit beamforming design can be cast into a quadratic optimization problem defined on a complex Stiefel manifold. Based on the majorization–minimization (MM) method, an iterative algorithm is then developed to compute the optimal transmit beamforming design by solving a series of orthogonal Procrustes problems (OPPs) that admit closed-form optimal solutions. Numerical findings serve to validate the efficacy of our scheme. It is demonstrated that our approach can achieve at least 73% higher spectral efficiency than the existing methods in a high signal-to-noise ratio (SNR) regime.

1. Introduction

Ideally suited for sixth-generation (6G) mobile communication systems, integrated sensing and communication (ISAC) has gained growing attention due to its enormous potential in overcoming spectrum congestion, saving hardware costs, and enhancing system spectrum efficiency [1,2,3,4].

A joint design for the coexistence of radar and communication systems to achieve spectrum sharing has previously been used to enable ISAC in the same frequency band but on different devices. In this approach, radar and communication systems exchange side information for obtaining a beneficial cooperation. To this end, opportunistic spectrum sharing might be a directly viable scheme where the communication systems become active only if the spatial and spectral resources are not occupied by the radar [5]. However, this approach does not allow two systems to work simultaneously. As a step forward, the authors of [6] designed radar beamformers by taking the zero-forcing criterion, thus eliminating radar interference with communications. More recent beamforming designs exploit optimization tools to realize radar and communication coexistence. In [7,8], a radar beamformer and a communication covariance matrix were jointly designed with the aim of optimizing the signal-to-interference-plus-noise ratio (SINR) at the radar receiver under capacity and power constraints on the communication side. In a comparable scenario, efforts were made to robustly maximize radar detection probability in the presence of imperfect channel state information in [9]. Considering a more realistic scenario with surveillance radars deployed in cluttered environments, the mutual information between the input and output of a communication link [10] or communication energy efficiency [11] was optimized under radar performance and power budget constraints, respectively. Although these coexistence system designs seem an effective way to implement ISAC, they might entail extra expenses for hardware and energy, along with augmented system complexity [12].

To enhance efficiency, an encouraging strategy for enabling ISAC involves the creation of DFRC systems. These systems utilize identical signals for both target detection and user communication concurrently, operating on a unified platform. However, given the constrained resources of DFRC systems, ensuring or enhancing sensing and communication performance simultaneously becomes challenging. Essentially, stringent performance guarantees can only be afforded to one system, while the other system can only achieve a best-effort performance level.

The majority of prior studies [12,13,14,15,16,17,18] have focused on enhancing radar functionality while accommodating communication constraints through beamforming methodologies, thereby shaping a communication-centric DFRC system. For instance, efforts have been made to minimize errors in beampattern matching while guaranteeing a pre-defined level of SINR for each communication user, as demonstrated in [12,13,14]. Under a similar communication performance constraint, the optimization of the Cramér–Rao bound (CRB) for target estimation was pursued to directly assess sensing capabilities in [15,16,17]. Additionally, joint designs for transmit and receive beamforming were developed in [18] to maximize the SINR at the radar receiver while meeting spectral efficiency requirements. These communication-centric designs, however, prioritize communication performance over sensing, so radar performance is much worse than it would be without communication functions. In order to ensure better sensing performance, the authors of [19] designed hybrid beamforming solutions to optimize the weighted sum of the communication spectral efficiency and the radar beampattern matching errors. It is possible to achieve the desired trade-off between communication and radar by adjusting the weight coefficients. Likewise, in [20], the receive SINR at the radar and multi-user interference were combined into a composite objective function, and the transmit and receive beamformers were designed jointly. Some recent designs tried to optimize the communication performance subject to the beampattern mismatch error constraint [21,22], which better preserved the sensing performance. However, these designs [19,20,21,22] still somewhat weaken the sensing performance compared with a separate radar without communication functions.

To avoid the performance loss of the radar, it is possible to incorporate communication functions into pre-designed radar systems, leading to a radar-centric DFRC system. Towards this goal, some earlier studies have tried to embed communication information into radar signals to achieve dual functionality. For example, by employing waveform diversity and sidelobe control, information sequences were embedded into radar pulses in [23]. Nevertheless, such information embedding designs cannot utilize the spatial diversity gains of multi-input–multi-output (MIMO). As MIMO radar sensing strongly relies on the transmit covariance of waveforms [24,25], some recent works optimized the communication performance under the prescribed transmit covariance constraint for stringent sensing quality assurance. Optimal transmit waveform/beamforming designs were proposed under the instantaneous covariance constraint in [26] or under the average covariance constraint in [27]. However, the approaches in [26,27] could not be extended to more general MIMO DFRC systems that admit transmissions of multiple data streams for each communication user. In view of this, our previous work [28] put forth efficient (linear) transmit and receive beamforming designs to aid in maximizing user rates under the prescribed transmission covariance constraint in MIMO DFRC systems.

The beamforming schemes in [28] can significantly improve the communication (i.e., spectral efficiency) performance of DFRC systems while strictly preserving the radar sensing performance. Yet, such a linear beamforming design cannot effectively eliminate the inter-user interference and interference between radar and communication signals, thereby hindering further improvements in the communication rates. To overcome this limitation, in this paper, we propose to adopt dirty paper coding (DPC) to perform effective interference (pre-)cancellation at the transmitter (i.e., the base station) of downlink communications and address optimal beamforming designs for a DPC-based MIMO DFRC system that detects several targets and communicates with a number of users simultaneously on the same hardware platform.

The key contributions of this work are summarized as follows:

• We put forth novel beamforming design approaches to achieve the maximum weighted communication sum rate with specified transmit covariance for a DPC-based MIMO DFRC system.
• We reformulate the intended problem and develop a block coordinate descent (BCD) method that calculates transmit and receive beamforming solutions in an alternating manner by leveraging the connection between the weighted sum rate and the weighted minimum mean squared error (MMSE).
• With this BCD approach, the optimal receive beamforming design can be obtained in closed form, and the optimal transmit beamforming design can be cast into a quadratic optimization problem defined on a complex Stiefel manifold. Based on the majorization–minimization (MM) approach, an iterative algorithm is then developed to compute the optimal transmit beamforming design by solving a series of orthogonal Procrustes problems (OPPs) that admit closed-form optimal solutions.

The subsequent sections of this paper are structured as follows: Section 2 delineates the system model. In Section 3, we introduce a proficient BCD algorithm aimed at attaining a high-performance beamforming configuration. Section 4 presents simulation outcomes, succeeded by the conclusions outlined in Section 5.

Notation: Boldface upper- and lowercases stand for matrices and vectors, respectively; ${(·)}^T$, ${(·)}^H$, ${(·)}^{-1}$, $\mathrm{Tr}(·)$, $\det (·)$, and $\left|\right|·|{|}_F$ signify the transpose, conjugate transpose, inverse, trace, determinant, and Frobenius norm of a matrix, respectively; for a symmetric matrix $\mathbf{A}$, $\mathbf{A}⪰\mathbf{0}$ signifies that $\mathbf{A}$ is positive semi-definite; ${\mathbf{I}}_n$ stands for the $n\times n$ identity matrix; ${\mathbb{C}}^{n\times m}$ represents the set of $n\times m$ complex matrices; $\mathrm{Re}\{·\}$ stands for taking the real part of a complex variable, while $\mathbb{E}\{·\}$ stands for taking ensemble expectation; $\mathcal{CN}(·,·)$ represents the complex normal distribution.

2. System Model

We explore a MIMO DFRC setup where a base station (BS) concurrently detects J distant targets and communicates with K downlink users on the same hardware platform. We use Figure 1 to illustrate the setup. The BS is outfitted with $N_{\mathrm{tx}}$ transmitting antennas, while each user possesses $N_{\mathrm{rx}}$ ($N_{\mathrm{rx}}⩽N_{\mathrm{tx}}$) receiving antennas. We use ${\mathbf{H}}_k\in {\mathbb{C}}^{N_{\mathrm{rx}}\times N_{\mathrm{tx}}}$ to represent the channel matrix between the BS and user k. We assume that the perfect channel state information (CSI) is known both at the transmitting side and the receiving side.

Radar waveforms and communication symbols are precoded into the transmit signal $\mathbf{x}\left(n\right)$:

$$\mathbf{x}\left(n\right)={\mathbf{F}}_{\mathrm{c}}\mathbf{c}\left(n\right)+{\mathbf{F}}_{\mathrm{r}}\mathbf{r}\left(n\right),n=0,1,\cdots ,$$

(1)

where $\mathbf{c}\left(n\right)={[{\mathbf{c}}_1^T\left(n\right),\cdots ,{\mathbf{c}}_K^T\left(n\right)]}^T$ encompasses $D=d\times K$ streams of information, undergoing precoding through the utilization of the communication beamforming matrix ${\mathbf{F}}_{\mathrm{c}}=[{\mathbf{F}}_1,\cdots ,{\mathbf{F}}_K]\in {\mathbb{C}}^{N_{\mathrm{tx}}\times D}$. Here, ${\mathbf{F}}_k\in {\mathbb{C}}^{N_{\mathrm{tx}}\times d}$ is designated as the beamforming matrix for transmitting the (multi-stream) communication symbol vector ${\mathbf{c}}_k\left(n\right)\in {\mathbb{C}}^d$ to user k at time n from the BS. For sensing purposes, a radar signal vector $\mathbf{r}\left(n\right)$, comprising $(N_{\mathrm{tx}}-D)$ random symbols generated independently [27], is transmitted concurrently with $\mathbf{c}\left(n\right)$, and it undergoes precoding using a radar beamforming matrix ${\mathbf{F}}_{\mathrm{r}}\in {\mathbb{C}}^{N_{\mathrm{tx}}\times (N_{\mathrm{tx}}-D)}$.

It is assumed that the communication symbol vectors of different users are mutually independent, which means that $\mathbb{E}\left\{\mathbf{c}\left(n\right){\mathbf{c}}^H\left(n\right)\right\}={\mathbf{I}}_D$. Additionally, it is also assumed that the overall communication symbol vector is irrelevant to the radar signal vector, i.e., $\mathbb{E}\left\{\mathbf{r}\left(n\right){\mathbf{c}}^H\left(n\right)\right\}={\mathbf{0}}_{(N_{\mathrm{tx}}-D)\times D}$.

2.1. Sensing Performance Guarantee

For the signal model (1) under consideration, communication beamforming matrix ${\mathbf{F}}_{\mathrm{c}}$ can be used for the sensing purpose jointly with radar beamforming matrix ${\mathbf{F}}_{\mathrm{r}}$, since it is perfectly known at the BS. We denote the overall $N_{\mathrm{tx}}\times N_{\mathrm{tx}}$ beamforming matrix by $\mathbf{F}=[{\mathbf{F}}_{\mathrm{c}},{\mathbf{F}}_{\mathrm{r}}]$. The sensing performance of a MIMO radar is highly related to its transmit beampattern, which depends on the covariance of transmit signals, i.e.,

$${\mathbf{R}}_x=\mathbb{E}\left\{\mathbf{x}\left(n\right){\mathbf{x}}^H\left(n\right)\right\}=\mathbf{F}{\mathbf{F}}^H={\mathbf{F}}_{\mathrm{c}}{\mathbf{F}}_{\mathrm{c}}^H+{\mathbf{F}}_{\mathrm{r}}{\mathbf{F}}_{\mathrm{r}}^H.$$

(2)

To better preserve the sensing performance, a desired transmit covariance matrix (${\mathbf{R}}_{\mathrm{des}}$) should be designed in advance according to the specific application requirements [24]. Then, the transmit covariance is required to match the predetermined ${\mathbf{R}}_{\mathrm{des}}$, as given by

$$\mathbf{F}{\mathbf{F}}^H={\mathbf{R}}_{\mathrm{des}}.$$

(3)

By enforcing this transmit covariance constraint on $\mathbf{F}$, the sensing performance can be guaranteed. From this constraint, we can observe the importance of introducing radar waveforms precoded by ${\mathbf{F}}_{\mathrm{r}}$ in such a DFRC system. To be specific, if ${\mathbf{F}}_{\mathrm{r}}=\mathbf{0}$, in other words, only communication symbols are transmitted, the rank of ${\mathbf{R}}_x$ cannot exceed D with ${\mathbf{R}}_x={\mathbf{F}}_{\mathrm{c}}{{\mathbf{F}}_{\mathrm{c}}}^H$. As a result, the prescribed ${\mathbf{R}}_{\mathrm{des}}$ whose rank is larger than D can never be satisfied. With the introduced ${\mathbf{F}}_{\mathrm{r}}$, the degree of freedom (DoF) of ${\mathbf{R}}_x$ is equal to the number of transmitting antennas, $N_{\mathrm{tx}}$, and any prescribed ${\mathbf{R}}_{\mathrm{des}}$ can be met with an appropriate combination of ${\mathbf{F}}_{\mathrm{c}}$ and ${\mathbf{F}}_{\mathrm{r}}$.

It is noteworthy that the transmit covariance constraint (3) essentially imposes a limitation on the power allocation for the transmit beamforming matrix ($\mathbf{F}$). The overall transmit power can be explicitly determined as $P=\mathrm{Tr}\left(\mathbf{F}{\mathbf{F}}^H\right)=\mathrm{Tr}\left({\mathbf{R}}_{\mathrm{des}}\right)$ under a particular ${\mathbf{R}}_{\mathrm{des}}$. In conventional beamforming optimization scenarios, the power allocation restriction might manifest as $\mathrm{Tr}\left(\mathbf{F}{\mathbf{F}}^H\right)⩽P_{\max }$, where $P_{\max }$ denotes the maximum allowable power budget at the BS. In our DFRC system, as the radar always probes signals with the maximal available power, we should always have $\mathrm{Tr}\left(\mathbf{F}{\mathbf{F}}^H\right)=P_{\max }$. Thus, the transmit power constraint on $\mathbf{F}$ can be implicitly incorporated into constraint (3) for compactness.

2.2. DPC for Multi-User Communications

Although the transmit covariance constraint (3) strictly preserves the sensing performance, it will, in turn, damage the performance of downlink multi-user communications. Specifically, to zero-force the interference from the other users’ signals and the radar signals, the transmit beamforming matrix ($\mathbf{F}$) should satisfy [29]

$${\mathbf{R}}_0={\mathbf{HFF}}^H{\mathbf{H}}^H,$$

(4)

where we have $\mathbf{H}=[{\mathbf{H}}_1^H,\cdots ,{\mathbf{H}}_K^H{]}^H$ and ${\mathbf{R}}_0$ is the block diagonal. Nevertheless, with $\mathbf{F}{\mathbf{F}}^H={\mathbf{R}}_{\mathrm{des}}$ in (3), the term ${\mathbf{HFF}}^H{\mathbf{H}}^H$ is fixed for the given channel matrix $\mathbf{H}$ and the pre-designed transmit covariance ${\mathbf{R}}_{\mathrm{des}}$, making it hard to meet (4). As a result, the interference cannot be pre-canceled effectively by the linear transmit beamforming design of $\mathbf{F}$.

To further improve the communication performance, we resort to the non-linear precoding technology, i.e., DPC, to eliminate the interference arising from the other users’ signals and the radar signals at the transmitter. From an information-theoretic perspective, if the transmitter is aware of the interference, the capacity of an additive white Gaussian noise (AWGN) channel in the presence of the interference is equal to that of an AWGN channel without interference [30]. The key to this interesting result is the use of DPC, which can effectively pre-cancel all the known mutual interference among the multiple data streams. Based on the principles of DPC, we can serially encode first the radar signals and then the communication users’ signals. After encoding radar signal $\mathbf{r}\left(n\right)$, the signal of each user is encoded in reverse order, i.e., from the K-th user to the first user. In this way, when performing DPC for the k-th user’s signal ${\mathbf{c}}_k\left(n\right)$, the signals $\mathbf{r}\left(n\right)$ and ${\mathbf{c}}_{k+1}\left(n\right),\cdots ,{\mathbf{c}}_K\left(n\right)$ have already been encoded, while ${\mathbf{c}}_1\left(n\right),\cdots ,{\mathbf{c}}_{k-1}\left(n\right)$ have not. Thus, the interference from the radar signals and users $(k+1),\cdots ,K$ is known, and the interference from users $1,\cdots ,(k-1)$ is unknown at the transmitter. The signal received by user k is written as

$${\overline{\mathbf{y}}}_k\left(n\right)=\underset{\mathrm{desired}\mathrm{signal}}{\underbrace{{\mathbf{H}}_k{\mathbf{F}}_k{\mathbf{c}}_k\left(n\right)}}+\underset{\mathrm{effective}\mathrm{interference}}{\underbrace{{\mathbf{H}}_k\sum _{i<k}{\mathbf{F}}_i{\mathbf{c}}_i\left(n\right)}}+\underset{\mathrm{known}\mathrm{interference}}{\underbrace{{\mathbf{H}}_k\sum _{i>k}{\mathbf{F}}_i{\mathbf{c}}_i\left(n\right)}}+\underset{\mathrm{known}\mathrm{interference}}{\underbrace{{\mathbf{H}}_k{\mathbf{F}}_{\mathrm{r}}\mathbf{r}\left(n\right)}}+\underset{\mathrm{effective}\mathrm{interference}}{\underbrace{{\mathbf{n}}_k\left(n\right)}},$$

(5)

where ${\mathbf{n}}_k\left(n\right)\in {\mathbb{C}}^{N_{\mathrm{rx}}}$ represents AWGN with zero mean and covariance matrix ${\sigma }^2{\mathbf{I}}_{N_{\mathrm{rx}}}$. The noise is assumed to be irrelevant to radar and communication signals. With the help of DPC, the known interference, i.e., the third and fourth terms on the right-hand side (RHS) of (5), can be eliminated at the receiver of user k. Thus, the effective received signal at user k consists of only the desired signal and the effective noise, as given by

$${\mathbf{y}}_k\left(n\right)={\mathbf{H}}_k{\mathbf{F}}_k{\mathbf{c}}_k\left(n\right)+{\mathbf{H}}_k\sum _{i<k}{\mathbf{F}}_i{\mathbf{c}}_i\left(n\right)+{\mathbf{n}}_k\left(n\right).$$

(6)

By using a linear reception beamforming matrix labeled ${\mathbf{G}}_k\in {\mathbb{C}}^{N_{\mathrm{rx}}\times d}$, the signal approximation at user k can be represented as

$${\widehat{\mathbf{c}}}_k\left(n\right)={\mathbf{G}}_k^H{\mathbf{y}}_k\left(n\right).$$

(7)

To evaluate the performance of downlink communications, the weighted sum rate is selected as the performance metric. For the DPC-based MIMO DFRC system under consideration, the weighted sum rate can be denoted by

$$C=\sum _{k=1}^K{\omega }_k{C}_k,$$

(8)

where weight ${\omega }_k$ is utilized to indicate the priority of user k in our system. Based on the signal model (6), the achievable rate ($C_k$) of k-th user can be calculated as

$$C_k=\log \det ({\mathbf{I}}_d+{\mathbf{F}}_k^H{\mathbf{H}}_k^H{({\sigma }^2{\mathbf{I}}_{N_{\mathrm{rx}}}+\sum _{i<k}{\mathbf{H}}_k{\mathbf{F}}_i{\mathbf{F}}_i^H{\mathbf{H}}_k^H)}^{-1}{\mathbf{H}}_k{\mathbf{F}}_k).$$

(9)

Our goal is to design the ${\mathbf{F}}_{\mathrm{c}}$ and ${\mathbf{F}}_{\mathrm{r}}$ that achieve the maximum weighted sum rate in (8) subject to the transmit covariance constraint (3) to guarantee the sensing performance preferentially.

3. Proposed Beamforming Design

Based on objective (8) and constraint (3), we wish to solve the following optimization problem:

$$\begin{array}{cc}\hfill \underset{\mathbf{F}}{max}& \sum _{k=1}^K{\omega }_k\log \det ({\mathbf{I}}_d+{\mathbf{F}}_k^H{\mathbf{H}}_k^H{({\sigma }^2{\mathbf{I}}_{N_{\mathrm{rx}}}+\sum _{i<k}{\mathbf{H}}_k{\mathbf{F}}_i{\mathbf{F}}_i^H{\mathbf{H}}_k^H)}^{-1}{\mathbf{H}}_k{\mathbf{F}}_k)\hfill \end{array}$$

(10a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& \mathbf{F}{\mathbf{F}}^H={\mathbf{R}}_{\mathrm{des}}.\hfill \end{array}$$

(10b)

Note that as explained in Section 2, the transmit power budget constraint has been incorporated into ${\mathbf{R}}_{\mathrm{des}}$ in (10b). Clearly, problem (10) exhibits non-convexity arising from its non-convex objective and quadratic equality constraint. In what follows, we put forth a BCD-type algorithm to solve this problem.

3.1. Problem Reformulation

To facilitate an efficient solver for (10), we transform the non-convex objective (10b) into an equivalent form based on the relation between the weighted sum rate and the weighted MMSE. This is performed by first using $\mathbf{G}=[{\mathbf{G}}_1,\dots ,{\mathbf{G}}_K]$ to collect all receive beamforming matrices of all downlink users. The MSE matrix of k-th user can be written as

$$\begin{array}{c}{\mathbf{E}}_k(\mathbf{G},\mathbf{F})=\mathbb{E}\left\{({\widehat{\mathbf{c}}}_k\left(t\right)-{\mathbf{c}}_k\left(t\right)){({\widehat{\mathbf{c}}}_k\left(t\right)-{\mathbf{c}}_k\left(t\right))}^H\right\}\hfill \\ ={\mathbf{I}}_d-2Re\left\{{\mathbf{G}}_k^H{\mathbf{H}}_k{\mathbf{F}}_k\right\}+{\displaystyle \sum _{i⩽k}}{\mathbf{G}}_k^H{\mathbf{H}}_k{\mathbf{F}}_i{\mathbf{F}}_i^H{\mathbf{H}}_k^H{\mathbf{G}}_k+{\sigma }^2{\mathbf{G}}_k^H{\mathbf{G}}_k.\hfill \end{array}$$

(11)

The original problem can then be reformulated into a matrix weighted sum–MSE problem:

$$\begin{array}{cc}\hfill \underset{\mathbf{F},\mathbf{G},\mathbf{W}}{min}& \sum _{k=1}^K{\omega }_k(\mathrm{Tr}\left({\mathbf{W}}_k{\mathbf{E}}_k\right)-\log \det \left({\mathbf{W}}_k\right))\hfill \end{array}$$

(12a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& \mathbf{F}{\mathbf{F}}^H={\mathbf{R}}_{\mathrm{des}},\hfill \end{array}$$

(12b)

where we introduce the weight matrices ${\mathbf{W}}_k⪰\mathbf{0}$, $\forall k$, as auxiliary optimization variables. It is observed that (12) consists of variables $\mathbf{F}$, $\mathbf{G}$, and $\mathbf{W}$. The equivalence between problems (10) and (12) can be established by the introduced lemma [31,32].

Lemma 1. 

The optimal transmit beamforming solutions (${\mathbf{F}}^{*}$) to the problems in (10) and (12) coincide. Furthermore, ${\mathbf{F}}^{*}$ stands as a stationary point solution for (10) only if it constitutes a stationary point solution $({\mathbf{F}}^{*},{\mathbf{G}}^{*},{\mathbf{W}}^{*})$ for (12).

Based on Lemma 1, a BCD-type method can be readily developed as follows.

3.2. BCD Method

Since the optimization variables in (12) are composed of $\mathbf{F}$, $\mathbf{G}$, and $\mathbf{W}$, we may optimize them in an alternating manner. The whole procedure includes the following steps.

(1) Optimizing $\mathbf{G}$ with fixed $\mathbf{F}$ and $\mathbf{W}$

Solving ${\mathbf{G}}_k$ for (12) is equivalent to minimizing $\mathrm{Tr}\left({\mathbf{W}}_k{\mathbf{E}}_k\right)$. Here, the positive semi-definite ${\mathbf{W}}_k$ only serves as a coefficient matrix. Hence, minimizing $\mathrm{Tr}\left({\mathbf{W}}_k{\mathbf{E}}_k\right)$ is equivalent to minimizing $\mathrm{Tr}\left({\mathbf{E}}_k\right)$. Given $\mathbf{F}$ in (11), it is clear that the MMSE receive beamformer should be adopted by user k, i.e.,

$${\mathbf{G}}_k^{\mathrm{mmse}}={(\sum _{i⩽k}{\mathbf{H}}_k{\mathbf{F}}_i{\mathbf{F}}_i{\mathbf{H}}_k^H+{\sigma }^2{\mathbf{I}}_{N_{\mathrm{rx}}})}^{-1}{\mathbf{H}}_k{\mathbf{F}}_k,\forall k.$$

(13)

(2) Optimizing $\mathbf{W}$ with fixed $\mathbf{F}$ and $\mathbf{G}$

By substituting (13) into (11), the MSE matrix of user k now becomes

$$\begin{array}{c}\hfill {\mathbf{E}}_k^{\mathrm{mmse}}={\mathbf{I}}_d-{\mathbf{F}}_k^H{\mathbf{H}}_k^H{(\sum _{i⩽k}{\mathbf{H}}_k{\mathbf{F}}_i{\mathbf{F}}_i{\mathbf{H}}_k^H+{\sigma }^2{\mathbf{I}}_{N_{\mathrm{rx}}})}^{-1}{\mathbf{H}}_k{\mathbf{F}}_k.\end{array}$$

(14)

With fixed $\mathbf{F}$ and $\mathbf{G}$, we have an unconstrained convex problem regarding $\mathbf{W}$, to which the optimal solution is

$${\mathbf{W}}_k^{*}={\left({\mathbf{E}}_k^{\mathrm{mmse}}\right)}^{-1},\forall k.$$

(15)

This is obtained based on the first-order optimality condition of ${\mathbf{W}}_k$.

(3) Optimizing $\mathbf{F}$ with fixed $\mathbf{W}$ and $\mathbf{G}$

The most challenging phase of the suggested BCD strategy to tackle (12) involves the optimization of the transmit beamforming matrix, $\mathbf{F}$. The subproblem for $\mathbf{F}$ can be written as

$$\begin{array}{cc}\hfill \underset{\mathbf{F}}{min}& \sum _{k=1}^K{\omega }_k[\mathrm{Tr}({\mathbf{W}}_k{\mathbf{G}}_k^H{\mathbf{H}}_k\sum _{i⩽k}{\mathbf{F}}_i{\mathbf{F}}_i^H{\mathbf{H}}_k^H{\mathbf{G}}_k)-2Re\{\mathrm{Tr}({\mathbf{W}}_k{\mathbf{G}}_k^H{\mathbf{H}}_k{\mathbf{F}}_k)\}]\hfill \end{array}$$

(16a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& \mathbf{F}{\mathbf{F}}^H={\mathbf{R}}_{\mathrm{des}}.\hfill \end{array}$$

(16b)

Obviously, (16) is a non-convex problem stemming from the quadratic equality constraint in (16b). Unlike (13) and (15), a closed-form optimal solution cannot be derived for (16). In what follows, we put forward an iterative algorithm to resolve this problem efficiently.

3.3. Optimization of Transmit Beamforming Matrix $\mathbf{F}$

To solve (16), we first transform it into a quadratic problem defined on a complex Stiefel manifold. There is indeed inherent orthogonality among the transmit beamforming vectors due to the transmit covariance constraint in (16b). To see it, Cholesky decomposition is applied to ${\mathbf{R}}_{\mathrm{des}}$, i.e.,

$${\mathbf{R}}_{\mathrm{des}}=\mathbf{L}{\mathbf{L}}^H,$$

(17)

where $\mathbf{L}\in {\mathbb{C}}^{N_{\mathrm{tx}}\times N_{\mathrm{tx}}}$ is a lower triangular matrix.

We substitute (17) into (16b) and then rewrite (16b) as

$${\mathbf{L}}^{-1}\mathbf{F}{\mathbf{F}}^H{\mathbf{L}}^{-H}={\mathbf{I}}_{N_{\mathrm{tx}}}.$$

(18)

With $\tilde{\mathbf{F}}={\mathbf{L}}^{-1}\mathbf{F}$ and ${\tilde{\mathbf{H}}}_k={\mathbf{H}}_k\mathbf{L}$ for brevity of notation, we can equivalently rewrite (16) as

$$\begin{array}{cc}\hfill \underset{\tilde{\mathbf{F}}}{min}& \sum _{k=1}^K{\omega }_k[\mathrm{Tr}({\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k\sum _{i⩽k}{\tilde{\mathbf{F}}}_i{\tilde{\mathbf{F}}}_i^H{\tilde{\mathbf{H}}}_k^H{\mathbf{G}}_k)-2Re\{\mathrm{Tr}({\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k{\tilde{\mathbf{F}}}_k)\}]\hfill \end{array}$$

(19a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& \tilde{\mathbf{F}}{\tilde{\mathbf{F}}}^H={\mathbf{I}}_{N_{\mathrm{tx}}}.\hfill \end{array}$$

(19b)

Given $\tilde{\mathbf{F}}=[{\tilde{\mathbf{F}}}_{\mathrm{c}},{\tilde{\mathbf{F}}}_{\mathrm{r}}]$, we can obtain ${\tilde{\mathbf{F}}}_{\mathrm{c}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}={\mathbf{I}}_D$ and ${\tilde{\mathbf{F}}}_{\mathrm{r}}^H{\tilde{\mathbf{F}}}_{\mathrm{r}}={\mathbf{I}}_{N_{\mathrm{tx}}-D}$. Additionally, it is observed that objective (19a) is only a function of ${\tilde{\mathbf{F}}}_{\mathrm{c}}$ and that ${\tilde{\mathbf{F}}}_{\mathrm{r}}$ only contributes to meet constraint (19b). This is because the radar signals would not interfere with communication signals due to the use of DPC at the transmitter. For this reason, we can replace constraint (19b) with ${\tilde{\mathbf{F}}}_{\mathrm{c}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}={\mathbf{I}}_D$ by taking ${\tilde{\mathbf{F}}}_{\mathrm{c}}$ as the optimization variable, instead of $\tilde{\mathbf{F}}$.

We can also perform the following rearrangements on (19a)

$$\begin{array}{cc}\hfill & \sum _{k=1}^K{\omega }_k[\mathrm{Tr}({\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k\sum _{i⩽k}{\tilde{\mathbf{F}}}_i{\tilde{\mathbf{F}}}_i^H{\tilde{\mathbf{H}}}_k^H{\mathbf{G}}_k)-2Re\left\{\mathrm{Tr}\left({\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k{\tilde{\mathbf{F}}}_k\right)\right\}]\hfill \end{array}$$

(20a)

$$\begin{array}{cc}\hfill =& \sum _{k=1}^K\mathrm{Tr}({\omega }_k{\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k\sum _{i⩽k}{\tilde{\mathbf{F}}}_i{\tilde{\mathbf{F}}}_i^H{\tilde{\mathbf{H}}}_k^H{\mathbf{G}}_k)-\sum _{k=1}^{K}2Re\{\mathrm{Tr}({\omega }_k{\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k{\tilde{\mathbf{F}}}_k)\}\hfill \end{array}$$

(20b)

$$\begin{array}{cc}\hfill \stackrel{(a)}{=}& \sum _{k=1}^K\mathrm{Tr}(\sum _{i⩽k}{\tilde{\mathbf{F}}}_i^H{\omega }_k{\tilde{\mathbf{H}}}_k^H{\mathbf{G}}_k{\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k{\tilde{\mathbf{F}}}_i)-\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\omega }_k{\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k{\tilde{\mathbf{F}}}_k\right)\right\}\hfill \end{array}$$

(20c)

$$\begin{array}{cc}\hfill \stackrel{(b)}{=}& \sum _{k=1}^K\mathrm{Tr}({\tilde{\mathbf{F}}}_k^H(\sum _{i=k}^K{\omega }_i{\tilde{\mathbf{H}}}_i^H{\mathbf{G}}_i{\mathbf{W}}_i{\mathbf{G}}_i^H{\tilde{\mathbf{H}}}_i){\tilde{\mathbf{F}}}_k)-\sum _{k=1}^{K}2Re\{\mathrm{Tr}({\omega }_k{\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k{\tilde{\mathbf{F}}}_k)\},\hfill \end{array}$$

(20d)

where $\left(a\right)$ is based on $\mathrm{Tr}\left(\mathbf{A}\mathbf{B}\right)=\mathrm{Tr}\left(\mathbf{B}\mathbf{A}\right)$ and $\left(b\right)$ is obtained through polynomial rearrangements of the first term in (20c).

We define ${\mathbf{M}}_k={\displaystyle \sum _{i=k}^K}{\omega }_i{\tilde{\mathbf{H}}}_i^H{\mathbf{G}}_i{\mathbf{W}}_i{\mathbf{G}}_i^H{\tilde{\mathbf{H}}}_i$ and ${\mathbf{N}}_k={\omega }_k{\mathbf{W}}_k{\mathbf{G}}_k^H{\tilde{\mathbf{H}}}_k$ in (20); we can then equivalently rewrite (16) as

$$\begin{array}{cc}\hfill \underset{{\tilde{\mathbf{F}}}_{\mathrm{c}}}{min}& \sum _{k=1}^K\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{M}}_k{\tilde{\mathbf{F}}}_k\right)-\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\mathbf{N}}_k{\tilde{\mathbf{F}}}_k\right)\right\}\hfill \end{array}$$

(21a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& {\tilde{\mathbf{F}}}_{\mathrm{c}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}={\mathbf{I}}_D.\hfill \end{array}$$

(21b)

Notice that (21) is a quadratic problem defined on a complex Stiefel manifold, according to the non-convex orthogonality constraint in (21b). Although (21) is non-convex, an MM optimization approach can be employed to solve it iteratively by tackling a series of OPPs, with a closed-form solution per iteration.

To be specific, each iteration of the MM approach involves two procedures [33]: (i) (majorization) finding a surrogate function that provides a local upper bound to the objective function and (ii) (minimization) minimizing the surrogate function. By selecting an appropriate surrogate function, solving the original difficult problem can be replaced by solving a sequence of approximate problems that are easy to deal with. In the context of this paper, we can find a linear surrogate function for (21a) at each iteration of MM and transform each approximate problem per iteration into an OPP, resulting in a closed-form, globally optimal solution. The details are as follows.

First, we construct the surrogate function as follows: By denoting the current local point in the t-th iteration as ${\tilde{\mathbf{F}}}_{\mathrm{c}}^t$ and defining (21a) as $f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}\right)$, we have the following proposition.

Proposition 1. 

$f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}\right)$ is upper-bounded by a linear function $g\left(\widetilde{{\mathbf{F}}_{\mathrm{c}}}\right|{\tilde{\mathbf{F}}}_{\mathrm{c}}^t)$ at ${\tilde{\mathbf{F}}}_{\mathrm{c}}^t$, as given by

$$f\left(\widetilde{{\mathbf{F}}_{\mathrm{c}}}\right)⩽g\left(\widetilde{{\mathbf{F}}_{\mathrm{c}}}\right|{\tilde{\mathbf{F}}}_{\mathrm{c}}^t)\triangleq \sum _{k=1}^{K}2Re\left\{\mathit{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k^t\right)\right\}-\sum _{k=1}^{K}2Re\left\{\mathit{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{N}}_k^H\right)\right\}+\mathit{const},$$

(22)

where ${\mathbf{K}}_k\triangleq {\mathbf{M}}_k-{\lambda }_k^{max}{\mathbf{I}}_{N_{\mathrm{tx}}}$ and ${\lambda }_k^{max}$ is the largest eigenvalue of ${\mathbf{M}}_k$. We use “$\mathrm{const}$” to represent a constant term. Additionally, the equality is achieved at ${\tilde{\mathbf{F}}}_{\mathrm{c}}^t$.

Proof. 

Since the second term of $f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}\right)$, i.e., $-{\displaystyle \sum _{k=1}^K}2Re\left\{\mathrm{Tr}\left({\mathbf{N}}_k{\tilde{\mathbf{F}}}_k\right)\right\}$, is linear, we only need to construct a surrogate function for the first term of $f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}\right)$, i.e., ${\displaystyle \sum _{k=1}^K}\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{M}}_k{\tilde{\mathbf{F}}}_k\right)$. Since $\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{M}}_k{\tilde{\mathbf{F}}}_k\right)$ is convex with respect to ${\tilde{\mathbf{F}}}_k$, it cannot be readily upper-bounded by using its first-order Taylor’s expansion. Thus, we perform the following reformulations:

$$\begin{array}{cc}\hfill \mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{M}}_k{\tilde{\mathbf{F}}}_k\right)& =\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H({\mathbf{M}}_k-{\lambda }_k^{max}\mathbf{I}){\tilde{\mathbf{F}}}_k\right)+\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H\left({\lambda }_k^{max}\mathbf{I}\right){\tilde{\mathbf{F}}}_k\right)\hfill \end{array}$$

(23a)

$$\begin{array}{cc}\hfill & \stackrel{\left(a\right)}{=}\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k\right)+{\lambda }_k^{max}d,\hfill \end{array}$$

(23b)

where $\left(a\right)$ is based on ${\mathbf{K}}_k\triangleq {\mathbf{M}}_k-{\lambda }_k^{max}{\mathbf{I}}_{N_{\mathrm{tx}}}$ and $\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\tilde{\mathbf{F}}}_k\right)=d$ as implied by (21b).

Since ${\mathbf{K}}_k$ is now a negative semi-definite matrix, the term $\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k\right)$ is concave over ${\tilde{\mathbf{F}}}_k$. Thus, it is upper-bounded by its first-order Taylor’s expansion, as given by

$$\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k\right)⩽2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k^t\right)\right\}+\mathrm{const},$$

(24)

where the equality is taken at ${\tilde{\mathbf{F}}}_{\mathrm{c}}^t$.

Then, we have

$$\begin{array}{cc}\hfill f\left(\widetilde{{\mathbf{F}}_{\mathrm{c}}}\right)& =\sum _{k=1}^K\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{M}}_k{\tilde{\mathbf{F}}}_k\right)-\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{N}}_k^H\right)\right\}\hfill \end{array}$$

(25a)

$$\begin{array}{cc}\hfill & \stackrel{\left(a\right)}{=}\sum _{k=1}^K\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k\right)-\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{N}}_k^H\right)\right\}+\mathrm{const}\hfill \end{array}$$

(25b)

$$\begin{array}{cc}\hfill & \stackrel{\left(b\right)}{⩽}\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k^t\right)\right\}-\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{N}}_k^H\right)\right\}+\mathrm{const}\hfill \end{array}$$

(25c)

$$\begin{array}{cc}\hfill & =g\left(\widetilde{{\mathbf{F}}_{\mathrm{c}}}\right|{\tilde{\mathbf{F}}}_{\mathrm{c}}^t).\hfill \end{array}$$

(25d)

where $\left(a\right)$ is based on (23) and $\left(b\right)$ holds due to (24). The proposition then readily follows.    □

Based on Proposition 1, problem (21) is transformed into an approximate problem at point ${\tilde{\mathbf{F}}}_{\mathrm{c}}^t$ in the t-th iteration with the objective replaced by $g\left(\widetilde{{\mathbf{F}}_{\mathrm{c}}}\right|{\tilde{\mathbf{F}}}_{\mathrm{c}}^t)$, i.e.,

$$\begin{array}{cc}\hfill \underset{{\tilde{\mathbf{F}}}_{\mathrm{c}}}{min}& \sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k^t\right)\right\}-\sum _{k=1}^{K}2Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{N}}_k^H\right)\right\}+\mathrm{const}\hfill \end{array}$$

(26a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& {\tilde{\mathbf{F}}}_{\mathrm{c}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}={\mathbf{I}}_D.\hfill \end{array}$$

(26b)

For (26a), we drop the constant term and perform the following rearrangements:

$$\begin{array}{c}{\displaystyle \sum _{k=1}^K}Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{K}}_k{\tilde{\mathbf{F}}}_k^t\right)\right\}-{\displaystyle \sum _{k=1}^K}Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H{\mathbf{N}}_k^H\right)\right\}\hfill \\ ={\displaystyle \sum _{k=1}^K}Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_k^H\right({\mathbf{K}}_k{\tilde{\mathbf{F}}}_k^t-{\mathbf{N}}_k^H\left)\right)\right\}\hfill \\ =Re\left\{\mathrm{Tr}\left({\tilde{\mathbf{F}}}_{\mathrm{c}}^H[{\mathbf{K}}_1{\tilde{\mathbf{F}}}_1^t-{\mathbf{N}}_1^H,\cdots ,{\mathbf{K}}_K{\tilde{\mathbf{F}}}_K^t-{\mathbf{N}}_K^H]\right)\right\}.\hfill \end{array}$$

(27)

By defining $\mathbf{Q}=[-{\mathbf{K}}_1{\tilde{\mathbf{F}}}_1^t+{\mathbf{N}}_1^H,\cdots ,-{\mathbf{K}}_K{\tilde{\mathbf{F}}}_K^t+{\mathbf{N}}_K^H]$ and turning minimization into maximization, problem (26) can be rewritten as

$$\begin{array}{cc}\hfill \underset{{\tilde{\mathbf{F}}}_{\mathrm{c}}}{max}& \mathrm{Re}\left\{\mathrm{Tr}\right({{\tilde{\mathbf{F}}}_{\mathrm{c}}}^H\mathbf{Q}\left)\right\}\hfill \end{array}$$

(28a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& {{\tilde{\mathbf{F}}}_{\mathrm{c}}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}={\mathbf{I}}_D.\hfill \end{array}$$

(28b)

Interestingly, while problem (28) remains non-convex, it has an equivalent OPP reformulation as affirmed by the lemma below.

Lemma 2. 

Problem (28) can be equivalently transformed into the following OPP:

$$\underset{{\tilde{\mathbf{F}}}_{\mathrm{c}}}{min}\Vert \mathbf{Q}-{\tilde{\mathbf{F}}}_{\mathrm{c}}{\Vert }_F^2$$

(29a)

$$\begin{array}{cc}\hfill \mathrm{s.t.}& {\tilde{\mathbf{F}}}_{\mathrm{c}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}={\mathbf{I}}_D.\hfill \end{array}$$

(29b)

Proof. 

Notice that

$$\Vert \mathbf{Q}-{\tilde{\mathbf{F}}}_{\mathrm{c}}{\Vert }_F^2=\mathrm{Tr}({\mathbf{Q}}^H\mathbf{Q}+{\tilde{\mathbf{F}}}_{\mathrm{c}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}-2Re\left\{{\tilde{\mathbf{F}}}_{\mathrm{c}}^H\mathbf{Q}\right\}).$$

(30)

In (30), $\mathbf{Q}$ is fixed in each iteration, and $\mathrm{Tr}\left({\mathbf{Q}}^H\mathbf{Q}\right)$ is thus a constant. Additionally, we also have $\mathrm{Tr}\left({{\tilde{\mathbf{F}}}_{\mathrm{c}}}^H{\tilde{\mathbf{F}}}_{\mathrm{c}}\right)=D$ by constraint (29b). It then readily follows that the majorization of $\mathrm{Re}\left\{\mathrm{Tr}\right({{\tilde{\mathbf{F}}}_{\mathrm{c}}}^H\mathbf{Q}\left)\right\}$ is equivalent to the minimization of $\Vert \mathbf{Q}-{\tilde{\mathbf{F}}}_{\mathrm{c}}{\Vert }_F^2$.    □

Referring to Proposition 7 of [34], problem (29) indeed has a closed-form, globally optimal solution. To be specific, we perform SVD on matrix $\mathbf{Q}$, i.e., $\mathbf{Q}={\mathbf{U}}_{\mathrm{Q}}{\Sigma }_{\mathrm{Q}}{\mathbf{V}}_{\mathrm{Q}}^H$, where ${\Sigma }_{\mathrm{Q}}$ collects the eigenvalues of $\mathbf{Q}$ along its diagonal in descending order. Then, we can achieve the unique optimal solution for (29), as given by

$${\tilde{\mathbf{F}}}_{\mathrm{c}}^{*}={\mathbf{U}}_{\mathrm{Q}}(1:D){\mathbf{V}}_{\mathrm{Q}}^H,$$

(31)

where ${\mathbf{U}}_{\mathrm{Q}}(1:D)$ consists of the first D columns of ${\mathbf{U}}_{\mathrm{Q}}$.

Note that the optimal ${\tilde{\mathbf{F}}}_{\mathrm{c}}^{*}$ is also the optimal solution for (26) based on the equivalence between (26) and (29). Having obtained the optimal ${\tilde{\mathbf{F}}}_{\mathrm{c}}^{*}$ for (26), at the t-th iteration, we can update the current local point as

$${\tilde{\mathbf{F}}}_{\mathrm{c}}^{t+1}={\tilde{\mathbf{F}}}_{\mathrm{c}}^{*},$$

(32)

and repeat the process.

The convergence of the proposed MM algorithm can be guaranteed, since the whole procedure generates a sequence ${\left\{{\tilde{\mathbf{F}}}_{\mathrm{c}}^t\right\}}_{t=0}^{\infty }$ that monotonically decreases the objective value of $f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}\right)$ in (21a), i.e.,

$$f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}^{t+1}\right)⩽g\left({\tilde{\mathbf{F}}}_{\mathrm{c}}^{t+1}\right|{\tilde{\mathbf{F}}}_{\mathrm{c}}^t)⩽g\left({\tilde{\mathbf{F}}}_{\mathrm{c}}^t\right|{\tilde{\mathbf{F}}}_{\mathrm{c}}^t)=f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}^t\right),$$

(33)

and $f\left({\tilde{\mathbf{F}}}_{\mathrm{c}}\right)$ is lower-bounded over a compact feasible set. Upon convergence, the optimal radar beamforming matrix, ${\tilde{\mathbf{F}}}_{\mathrm{r}}^{*}$, should be projected into the null space of ${\tilde{\mathbf{F}}}_{\mathrm{c}}^{*}$ by constraint (19b). Based on the SVD of $\mathbf{Q}=[-{\mathbf{K}}_1{\tilde{\mathbf{F}}}_1^{*}+{\mathbf{N}}_1^H,\cdots ,-{\mathbf{K}}_K{\tilde{\mathbf{F}}}_K^{*}+{\mathbf{N}}_K^H]$, it can be readily inferred that

$${\tilde{\mathbf{F}}}_{\mathrm{r}}^{*}={\mathbf{U}}_{\mathrm{Q}}(D+1:N_{\mathrm{tx}}),$$

(34)

where ${\mathbf{U}}_{\mathrm{Q}}(D+1:N_{\mathrm{tx}})$ consists of the last $(N_{\mathrm{tx}}-D)$ column vectors of ${\mathbf{U}}_{\mathrm{Q}}$. Then, we have ${\tilde{\mathbf{F}}}^{*}=[{\tilde{\mathbf{F}}}_{\mathrm{c}}^{*},{\tilde{\mathbf{F}}}_{\mathrm{r}}^{*}]$, which, in turn, leads to ${\mathbf{F}}^{*}=\mathbf{L}{\tilde{\mathbf{F}}}^{*}$. As the ${\tilde{\mathbf{F}}}_{\mathrm{c}}^{*}$ obtained by the MM method is a stationary point solution for problem (21) [35], ${\mathbf{F}}^{*}$ is also a stationary point for the original subproblem (16) by the equivalence between the problems in (16) and (21).

3.4. Analysis of Overall Algorithm

The overall BCD procedure for solving (12) is formally described in Algorithm 1.

Algorithm 1 The proposed BCD algorithm for solving (12).

1: Initialize $\mathbf{F}=\mathbf{L}$. 2: Repeat 3:     Given $\mathbf{F}$, obtain ${\mathbf{G}}_k$ by (13), $\forall k$. 4:     Given $\mathbf{F}$ and $\mathbf{G}$, obtain ${\mathbf{W}}_k$ by (15), $\forall k$. 5:     Given $\mathbf{G}$ and $\mathbf{W}$, obtain $\tilde{\mathbf{F}}$ using MM approach and update $\mathbf{F}=\mathbf{L}\tilde{\mathbf{F}}$. 6: Until convergence.

The convergence of Algorithm 1 can be guaranteed, since each iteration decreases objective (12a) with a lower bound resulting from (12b). Additionally, objective (12a) is differentiable and the constraint set that can be decoupled among $\mathbf{F}$, $\mathbf{G}$, and $\mathbf{W}$ with independent subproblems in (12). It readily follows from [36] that BCD-based Algorithm 1 certainly converges to at least a stationary point $({\mathbf{F}}^{*},{\mathbf{G}}^{*},{\mathbf{W}}^{*})$ of problem (12). By the equivalence between problems (10) and (12) established in Lemma 2, we immediately deduce that the corresponding ${\mathbf{F}}^{*}$ at point $({\mathbf{F}}^{*},{\mathbf{G}}^{*},{\mathbf{W}}^{*})$ is also a stationary point for the original problem (10).

With the optimal transmit beamforming matrix ${\mathbf{F}}^{*}$ obtained by Algorithm 1, to achieve the corresponding maximum rate at user k, the optimal receive beamforming matrix should adopt the classic MMSE form, as given by (cf. (13))

$${\mathbf{G}}_k^{\mathrm{mmse}}={(\sum _{i⩽k}{\mathbf{H}}_k{\mathbf{F}}_i{\mathbf{F}}_i{\mathbf{H}}_k^H+{\sigma }^2{\mathbf{I}}_{N_{\mathrm{rx}}})}^{-1}{\mathbf{H}}_k{\mathbf{F}}_k.$$

(35)

For the subproblem at each step of the proposed BCD algorithm, we either obtain a closed-form solution (cf. (13) or (15)) or iteratively solve it by using the MM method with a closed-form solution available per iteration. This leads to fast convergence to a high-quality beamforming design, as also verified numerically in Section 4.

The complexity of the proposed BCD is dominated by the optimization of $\mathbf{F}$ based on the MM method, i.e., Step 5 in Algorithm 1. Each iteration of MM for computing the optimal $\mathbf{F}$ only involves an SVD of $\mathbf{Q}$ and matrix multiplications. Hence, the per-iteration complexity of MM is $\mathcal{O}\left(N_{\mathrm{tx}}{D}^2\right)$. Let us suppose that the numbers of iterations before convergence of the MM method and the overall BCD are $T_1$ and $T_2$, respectively. The overall complexity of BCD-based Algorithm 1 is $\mathcal{O}\left(T_2{T}_1{N}_{\mathrm{tx}}{D}^2\right)$. As a result, BCD-based Algorithm 1 has (low) polynomial-time computational complexity.

One limitation of the proposed approach is that the implementation of the DPC scheme at the transmitter could increase the complexity of multi-user MIMO DFRC systems to some extent. Additionally, compared with the previous work [28], which admits a closed-form solution for globally optimal transmit beamforming, we adopt the MM method to optimize transmit beamforming $\mathbf{F}$ iteratively, leading to extra computational complexity.

4. Numerical Results

A Monte Carlo simulation is performed to assess the proposed scheme. Let us suppose that the BS and the communicating users are both equipped with uniform linear arrays with even antenna spacing. The spacing between adjacent antennas is set to half wavelength. By default, there are $N_{\mathrm{tx}}=16$ antennas at the BS. Let us assume that the same number of receiving antennas is used for radar echo signals. Channel fading model ${\mathbf{H}}_k,\forall k$, is based on Rayleigh fading; in other words, each element in ${\mathbf{H}}_k$ obeys the standard complex Gaussian distribution $\mathcal{CN}(0,1)$. The BS performs full-power transmissions with the total transmit power budget being P, and the transmit signal-to-noise-ratio (SNR) is defined as $P/{\sigma }^2$.

For the purpose of sensing, we suppose that the radar system detects $J=3$ targets of interest with angles of ${\theta }_1=-{60}^{\circ }$, ${\theta }_2=0^{\circ }$, and ${\theta }_3={60}^{\circ }$. We can design the prescribed transmit covariance matrix (${\mathbf{R}}_{\mathrm{des}}$) by solving a classic least-squares problem to minimize the mismatch errors between the designed radar beampattern and the ideal radar beampattern, as in [24]. The ideal beampattern (${\tilde{P}}_d\left(\theta \right)$) composed of three main beams with a beam width of $\Delta =9^{\circ }$ each is given by

$${\tilde{P}}_d\left(\theta \right)=\left\{\begin{array}{c}1,\mathrm{if}{\theta }_j-\Delta /2⩽\theta ⩽{\theta }_j+\Delta /2,j=1,2,3;\hfill \\ 0,\mathrm{otherwise}.\hfill \end{array}\right.$$

The designed radar beampattern is illustrated in Figure 2 with $N_{\mathrm{tx}}=16$.

To compare with “Proposed BCD” solution yielded by Algorithm 1, we introduce the following benchmark schemes:

• Dual program [27]: DPC is also utilized for the sum rate maximization by using the downlink–uplink duality scheme, which can achieve a globally optimal solution. Nevertheless, this scheme only applies to multi-input–single-output (MISO) systems, supporting transmissions of the single data stream to each user. To ensure a fair comparison, when each user possesses multiple receiving antennas, the row with the largest 2-norm in the corresponding channel matrix is selected for implementing this scheme.
• Without DPC [28]: Here, beamforming designs are put forth for MIMO DFRC systems without DPC at the transmitter. A BCD-type method is used to optimize the sum rate of communication users.
• Cholesky: The transmission beamforming matrix is designated as $\mathbf{F}=\mathbf{L}$, where $\mathbf{L}$ is derived through Cholesky decomposition applied to ${\mathbf{R}}_{\mathrm{des}}$, as defined in (17). It is worth mentioning that $\mathbf{L}$ also acts as the starting point in our proposed BCD approach.

Let us suppose that there exist $K=4$ users, each equipped with $N_{\mathrm{rx}}=2$ receiving antennas. Unless otherwise specified, it is assumed that each user has the same priority in the system, i.e., ${\omega }_k=1,\forall k$. The BS transmits $d=2$ data streams to each user. Figure 3 illustrates the spectral efficiency (the achievable sum rate under the unit bandwidth) of various beamforming strategies across different transmit SNRs. The results demonstrate the notable superiority of the proposed BCD scheme over alternative benchmark approaches. To be specific, the proposed design performs much better (e.g., 128% higher spectral efficiency at a 30 dB SNR) than the scheme in [28] without DPC at the transmitter, especially in a high SNR regime. Since the transmit beamforming matrix is confined by the transmit covariance constraint in (3), the interference from the other users’ signals and the radar signals cannot be zero-forced effectively if DPC is not performed at the transmitter, which then limits the achievable data rate of each user.

On the contrary, utilizing DPC at the transmitter enables interference cancellation even within the confines of the transmit covariance constraint, resulting in significantly enhanced overall spectral efficiency. In comparison with the “dual program”, which only accommodates $d=1$ data stream for each user, our proposed BCD solution can achieve a minimum of 73% higher spectral efficiency. This validates the efficacy of our approach in leveraging MIMO transmissions to augment communication capacity through the joint design of transmit and receive beamforming. Intriguingly, when we execute the proposed algorithm with $d=1$, the attained spectral efficiency closely matches the globally optimal performance obtained by the “dual program”, suggesting that our scheme may offer a near-optimal beamforming solution in multi-user scenarios. Moreover, the notable contrast in performance, exemplified by approximately a 20-fold increase in spectral efficiency at a 30 dB SNR, between our suggested BCD approach and Cholesky decomposition highlights the considerable advantages stemming from our BCD iterations.

Figure 4 plots spectral efficiency against the number of users under different schemes, where we set $N_{\mathrm{rx}}=2$ and SNR $=20$ dB. It is shown that the proposed BCD scheme obtains the best performance. As the number of users increases in the DFRC system, we observe that the overall spectral efficiency improves more if DPC is adopted at the transmitter. This is due to the fact that the transmit beamforming matrix is confined by the transmit covariance constraint, and the interference among different users cannot be eliminated effectively by linear precoding technology. Hence, when more users are introduced into the system, the interference imposed on other users increases significantly, leading to worse performance than the DPC-based scheme.

Figure 5 illustrates spectral efficiency at varying transmit SNRs alongside diverse configurations of transmitting and receiving antennas. Enhanced spectral efficiency is observed when each user is outfitted with additional receiving antennas, owing to the provision of heightened diversity gain. Nevertheless, in the absence of DPC at the transmitter, spectral efficiency experiences only marginal augmentation with an escalation in the number of transmitting antennas, particularly notable when $N_{\mathrm{rx}}=2$. This phenomenon stems from the transmit covariance constraint hindering the effective utilization of the supplementary degrees of freedom introduced by an expanded number of transmitting antennas. Conversely, for the DPC-based DFRC system, discernible enhancements in communication performance are apparent with an increase in either transmitting or receiving antennas.

Figure 6 shows the convergence behavior of the proposed algorithm (Algorithm 1) under different transmit SNRs with $d=2$, $K=4$, and $N_{\mathrm{rx}}=2$. It is evident that the proposed BCD scheme converges within only a few iterations, especially in a low SNR regime. To be specific, when SNR = 10 dB or 20 dB, the algorithm converges within about ten iterations. When SNR = 30 dB, as much as 85% of the converged spectral efficiency can be obtained after ten iterations.

5. Conclusions

Under a prescribed transmit covariance constraint, we developed the optimal beamforming design for MIMO DFRC systems that use DPC. A new BCD-type algorithm was employed to iteratively compute the transmit beamforming and the receive beamforming solutions. Under this approach, the optimal receive beamforming aligned with the classic MMSE approach, and the optimal transmit beamforming was obtained by solving a quadratic optimization problem defined on a complex Stiefel manifold. The overall algorithm converged fast to a high-quality beamforming design with low computational complexity. Simulation results demonstrated the superiority of the proposed scheme over the existing benchmarks, with at least 73% higher spectral efficiency in a high-SNR regime. It was also shown that a minimum of 85% of the converged spectral efficiency could be obtained after ten iterations.