Data were collected using the SAS in several instances in order to build a database of spectra to perform statistical spectrum analysis. To gain perspective on the spectrum, an ambient data collection was performed. Next, closed loop testing was performed in which an arbitrary waveform generator (AWG) fed spectra directly into the SAS via cable. Closed loop testing was performed using a pseudo-random generation technique, which relied on a combination of quantitative and qualitative observations of the ambient spectrum data.
5.2. Statistical Analysis of the Ambient Data
A statistical analysis of the spectra from the ambient data collection was performed to guide the generation of spectra for closed loop testing. Two statistical metrics were considered: PO and total signal energy (or alternately, total average power
) and PO, as derived in
Section 2.1 and
Section 2.2, respectively.
In
Section 2.1,
is calculated on the basis of a threshold,
. It should be noted that the threshold
is used to characterize the collected data in terms of total average power versus percent occupancy. This characterization provides an assessment of the spectra processed by the multi-objective optimization algorithms (discussed below). For example, the algorithms would process high-power, narrowband interference different from low-power, wideband interference. The functionality of the multi-objective optimization algorithms themselves do not use thresholding since the algorithms search a solution space for the result.
Ambiguity in the determination of PO stems from the method of selection of
. Several active noise floor estimation techniques can be used in defining
. For the purpose of characterizing these spectra, the luxury of real-time noise floor computation was not afforded. Rather, the SAS was taken off-line and calibration data were collected. A 50-Ω termination was connected to the input of the front end of the SAS and a relatively long-term run sequence was executed. A measurement of the system’s noise floor over time was effectively taken. A point-by-point average of the calibration data in the linear frequency domain was then calculated at each tuning frequency in order to provide an estimate of the noise floor mean. The point-by-point average is calculated using
where
is the total number of collections at each tuning frequency,
is the matrix of linear frequency domain data that is defined by the frequency bin,
, and the collection number,
.
The PO threshold,
, was then specified as
where
is the standard deviation of the point-by-point average. Setting
to three standard deviations above the running average is appropriate because only 0.1% of AWGN will exceed
.
Figure 21 is a scatter plot of the total average power versus PO for the ambient data collected. This scatter plot breaks the points up into select sub-bands showing the individual distributions for specific center frequencies (CF). These sub-band distributions can therefore be cataloged based on FCC mandated applications. For instance, in
Figure 21, there is a grouping of points around
= (−67 dBm, 45%) from a 50-MHz BW with a 1960-MHz CF. The corresponding allocated applications for a band of 1930 to 2165 MHz pertain to personal communication systems (PCS), 2-GHz mobile satellite services (MSS) uplink, space, fixed, and mobile. A combination of relatively high total average power and PO would support the assumption that these spectra are likely to contain one or more wideband signals. Due to the suburban location of the data collection, it can be surmised that cellular phones are the primary sources of the energy detected in this spectra, particularly Long-Term Evolution (LTE) mobile communication. A similar process can be used with the other sub-band distributions to speculate on the applications detected in the spectra.
Total average power and PO are not directly proportional; however the scattering is not completely random either. In many scenarios, a higher total average power will lead to a higher PO, although that is not always the case.
5.3. Closed Loop Testing
A pseudo-random database of spectra was compiled using the Agilent M8190A 12GSa/s Arbitrary Waveform Generator (AWG). The motivation for using a pseudo-random database of spectra is to attain an understanding as to the performance of the algorithms. A controlled spectra at the input of the SAS limits the complexity of the problem space, allowing for a more conclusive analysis as to the performance of the algorithms discussed. The signals produced by the AWG consisted of a pseudo-random combination of chirps (wide band) and sinusoids (narrow band).
During the closed loop testing, the spectra generated by the AWG were split into the SAS, LeCroy’s WaveMaster 8620A oscilloscope, and Tektronix RSA6114A real-time spectrum analyzer (RTSA). Only the data collected by the SAS are analyzed herein as the other systems were used for verification or separate testing purposes. Total average power and PO serve as the two input parameters that provide a basis for the generation of the pseudo-random spectra. These scatter plots presented in
Figure 21 serve as the foundation on which the pseudo-random data sets were modeled. An analysis as to the distribution of the total average power and PO of each band (denoted by its specific CF) was performed to select spectra with similar distributions from the pseudo-randomly generated database. The range of acceptable total average power and PO for each ambient spectra is displayed in
Figure 22 and
Table 2 along with the corresponding waveforms (WF) selected from the pseudo-random database.
These total average power and PO ranges were calculated as
and
respectively. Due to the extensive collection time and static nature of certain ambient spectra, the calculated ranges of total average power and PO had very small standard deviations making it difficult to find pseudo-randomly generated spectra that fell within the allotted ranges. In these circumstances, spectra with total average power and PO contiguous to the specified ranges were selected.
Figure 23 shows the total average power versus PO scatter plot for the pseudo-randomly generated waveforms. There is considerably less jitter in
Figure 23 as compared to that in
Figure 21 as the pseudo-random WFs are not time-varying.
5.5. Statistical Performance
The performance metrics utilized in this analysis are SINR, BW (as it affects radar range resolution), and percent interference (PI). SINR and BW are defined in
Section 2.4 and
Section 2.5, respectively. PI is defined as the ratio
where
is the number of emitters within the OSB selected by an algorithm and
is the number of emitters in the total view of the wideband spectrum. Because the spectra were generated, all of the signals within the spectra are known. Therefore, an emitter is defined as any individual sinusoid or chirp within the band of a spectrum. The determination as to whether an emitter is within a select OSB is binary. The FCC Spectrum Policy Task Force defines harmful interference as “Interference which endangers the functioning of a radionavigation service or of other safety services or seriously degrades, obstructs, or repeatedly interrupts a radiocommunication service operating in accordance with these [International] Radio Regulations” [
2]. The reasonable assumption is made that a radar transmission in an OSB overlapping less than 1% of a wideband signal does not degrade the primary user’s ability to perform its intended function. Therefore, any overlap of less than 1% is not considered interference. See
Figure 24 for an intuition regarding the bounds of a signal classified as an emitter. The noise floor of the spectrum in
Figure 24 is approximately −127 dBm. Any frequency bin with an amplitude above −127 dBm on the red line is declared an emitter. The PI quantifies the amount of co-channel interference that is caused by a radar transmission across the select OSB. An intuition for the spectral footprint, and to an extent the EMC, of a radar can be derived from PI.
An algorithm’s performance for each statistic is quantified through a root-mean-square error (RMSE) defined as
where
In Equation (35), is the metric’s best value, is the metric’s value as selected by the algorithm, and normalizes the error based on the total range of possible values. The best values () for PI, BW, and SINR are 0%, 35 MHz, and , respectively. A RMSE = 0 denotes a metric that has been optimized. In the case of BW, and . For PI, and .
Figure 25 shows the PI results for all of the algorithms for comparison with the PI for
, the maximum available BW yielding the best resolution. In
Figure 25, the 50 spectrum datum collected by the SAS were averaged for each WF.
Table 3 outlines the breakdown of emitters in each spectrum along with the average PI that is caused by the select OSB for each algorithm individually. Only once did all three algorithms (Waveform 3) have the optimal average PI of 0%, simultaneously resulting in an RMSE of 0. Of the three algorithms, the SS-BFE minimized PI most efficiently since its overall RMSE was the least, followed by SS-MO-BFE, and then the SS-MO whose RMSE was the highest. All three algorithms show a significant improvement over the PI of
, which has a constant PI of 100%.
In the spectra corresponding to Waveform 4, the RMSE of SS-BFE was observed to be slightly greater than that of the SS-MO-BFE. However, the difference is minute, indicating a slight, insignificant variation in performance. It is worth noting that both spectra contain a chirp emitter. In OSBs containing wideband signals such as chirps, the PSE value can be inflated as the energy within the wideband signal can be almost uniformly distributed leading to poor OSB identification. This is not typically the case as the energy in most wideband signals are less uniformly distributed than a sub-band containing purely noise. This effect is amplified in a controlled input scenario as signal fading is not a factor. The SINR weighting function in SS-MO-BFE counteracts the high PSE values as wideband signals are typically high powered RFI.
Figure 26 compares each algorithm’s
results averaged over each spectrum. Because range resolution is inversely related to BW, a high BW is desirable in order to achieve the best resolution.
Table 4 shows the average
value selected by each technique.
As anticipated, the SS-MO provides the best range resolution among all of the algorithms followed by the SS-MO-BFE, and then followed by SS-BFE over the complete dataset.
Of all three algorithms, the SS-MO provides the most weight for maximizing bandwidth as the weighting factor on the range resolution objective function, , is . This weighting factor is larger than that of the SS-MO-BFE, which uses a weighting factor of for . Because SS-BFE is not a traditional WSMO function, the weight applied to the range resolution objective cannot be directly compared to the other algorithms; however, the results are telling. The SS-BFE typically selects a much smaller OSB than the SS-MO or the SS-BFE as its main goal is to minimize co-channel interference.
Figure 27 displays the SINR results for all three algorithms for comparison, while
Table 5 reports the SINR performance of each algorithm for the twelve spectra.
Table 6 presents overall performance metrics averaged across all waveforms. The SINR results averaged over all spectra considered reveal that the SINR is highest for the SS-BFE, followed by SS-MO-BFE, SS-MO, and
, in that order. It should be noted, however, that the SINR values produced by each algorithm are relatively close as each algorithm maximizes SINR in its own manner. The principal goal of SS-BFE is to determine an OSB with minimal interference in which a radar can operate and SINR is inherently proportional to the inverse of interference power. Both SS-MO and SS-MO-BFE incorporate an SINR objective function. The significant SINR disparity can be seen between the
SINR values and the SINR values of the SS-MO, SS-MO-BFE, and SS-BFE OSBs. All three algorithms select an OSB with a considerable SINR improvement over the maximum BW.
5.6. Computational Performance
Although the OSB selection algorithms were not designed to optimize computational efficiency, the current computational expense of the algorithms is examined. It should be understood that this computational analysis is not indicative of the maximum attainable efficiency of the algorithm, but rather of the efficiency of each algorithm in its current state of being.
To quantify computational performance, the average processing time for each function is presented. These functions were called consecutively 100 times in a parent function. The input dataset consisted of decimated spectra composed of 2
10 or 1024 samples. The average computation time for each OSB function is outlined in
Table 7. The same processor and analysis platform were used when calculating the average computation time. Processing time is unique to processor speed and analysis platform, therefore the average processing time values presented in
Table 7 provide an understanding of the computational expense of each OSB function relative to the other two OSB functions. The least efficient prsssocess among all of the algorithms is computing the brute force PSE matrix. This computation may be able to be streamlined using more efficient matrix math; however, this solution has not been explored.
Another approach was recently investigated to refine the spectral content into a subset of representative information, thereby reducing the number of frequency bins (
) input to SS-MO [
14]. This fast SS-MO technique simultaneously reduces the computational complexity while preserving the performance of multi-objective optimization. Future research will investigate the utility of this approach with the proposed OSB selection techniques.