*1.2. Intra-Hour Solar Forecasting*

The energy alliance between UTSA and CPS Energy has as one of its primary goals the development of new solar forecasting technologies that combine inexpensive all-sky imaging cameras with sophisticated image processing techniques and artificial intelligence software to produce high-accuracy 15-min ahead solar irradiance forecasts. GHI consists of two components, the Direct Normal Irradiance (DNI) caused by sunlight traveling in a direct path from sun to PV array and the Diffuse Horizontal Irradiance (DHI), background illumination that is due to secondary reflections and absorption/re-radiation. The formula is *GHI* = cos(*θz*) *DNI* + *DHI* where *θ<sup>z</sup>* is the zenith angle. Shadows cast by low-level cumulus clouds significantly impact the DNI but have little effect upon the background DHI. While it is possible to predict DNI separately [21], for verifying PV power output forecasts GHI is used. Figure 2 displays the three quantities: GHI, DNI, and DHI, on the date 27 October 2015 at the NREL ESIF facility in Golden, CO. It shows that moderately cloudy conditions occasion multiple ramp events.

**Figure 2.** Daily solar irradiance (W/m2) for 27 October 2015, Golden, CO.

Figure 3 displays a sequence of eight pictures taken by the SkyImager at the NREL site, one every two minutes starting (upper left) at 12:31pm MST. At that time the sun is not obscured, but cumulus clouds are moving in from the left. At 12:35 the cloud begins to enter the solar disk and by 12:37 the sun is completely occluded. This continues until 12:44 when the cloud has moved past the sun and the DNI recovers. This ramp event is seen in the DNI oscillations that occur around the noon hour in Figure 2. While this example considers a single ramp event, it strongly suggests that the correlation between measured GHI and the presence of clouds obscuring the sun in the SkyImager pictures could be learned by AI models.

**Figure 3.** SkyImager sequence (27 October 2015, Golden, CO) showing a ramp event.

The challenge in short term prediction of PV power is simply "Where will cumulus cloud shadows be fifteen minutes from now?" Our approach incorporates as much of the physics as possible, but is an idealization necessitated by the requirement to produce GHI forecasts in real time for the MGMS. Solving Navier-Stokes for the true dynamics of the atmosphere is not feasible on a Raspberry Pi. If GHI can be accurately forecast, then predicting PV power output is straightforward. The evolution of clouds and irradiance shown in Figures 2 and 3 is even more striking when video of the images is viewed, confirming that SkyImager pictures are highly correlated with the observed GHI time series. Moreover, it suggests the camera sensor could be used to measure irradiance as well as predict it. Once GHI has been accurately forecast, it is usually straightforward to assign a corresponding PV power output, which is what the MGMS requires.

Figure 4 shows the relationship between GHI (Watts/m2) and PV power (Watts) from the RSF2 PV arrays located at NREL. The relationship is almost linear with a slight hysteresis effect that reflects the differences in morning versus afternoon irradiance. The task of predicting PV power output over multiple temporal and spatial scales, and from a variety of different equipment is a challenging one [22–24].

**Figure 4.** Relationship between GHI (W/m2) and PV Power (Watts) determined at NREL.

State of the Art in Solar Forecasting

As photovoltaics achieve greater penetration, the SmartGrid will demand accurate solar forecasting hence a network of low cost, distributed sensors to acquire large amounts of image and weather data for input to the forecasting algorithms. Commercial sky imaging systems often prove too costly and have proprietary software, leading several research groups to develop their own systems. The solar forecasting research group at UC San Diego [25–29] has done pioneering work in this field for many years. For one example, Coimbra et al. [30] proposed DNI forecasting models using images from a Yankee TSI 880 with hemispherical mirror as inputs to Artificial Neural Networks (ANN). The TSI has high capital and maintenance costs, uses a shadow band mechanism, and requires proprietary software. The UCSD Sky Imager described in Yang et al. [31] captures images with an upward-facing charge-coupled device (CCD) Panasonic sensor and a 4.5 mm focal length fisheye lens. Compared with the TSI it has higher resolution, greater dynamic range, and lossless PNG compression. The Universitat Erlangen-Nurnberg [32] group used a five-megapixel C-mount camera equipped with a fisheye lens. They implemented the Thirions "'daemons'" algorithm for image registration and cloud-motion estimation similar to optical flow. In Australia, West et al. [33] used off-the-shelf IP security cameras (Mobotix Q24, Vivotek FE8172V) for all-sky imaging. Inexpensive compared to the TSI, the cost of such systems still is ~800 euros. Rather than a feature-tracking strategy, they used dense optical flow to estimate cloud movement. See also Wood-Bradley [34]. Several research groups in China are working on the irradiance forecasting problem [35,36] generally using a TSI imager, but in one case Geostationary Statellite data [37]. As mentioned before dramatic improvements in GOES-R technology and resolution (spatial and temporal) will make this approach more attractive for intra-hour forecasting. See also the historical review [38] of irradiance and PV power forecasting that was produced using text mining and machine learning.

A recurring theme in the INTEGRATE project was that while all-sky imaging was a critical component of microgrid stability and control, it could not be developed in a stand-alone fashion but must be fully integrated into the microgrid management system (MGMS). Uriate et al. [39] discuss the importance of Ramp Rates (RR) on the inertial stability margin of a microgrid deployed at the Marine Corp Base at Twentynine Palms. They show that a large ramp in PV power can destabilize frequency when the PV load is suddenly transferred to the cogen. The inertia constant *H* of a generator is the ratio of stored kinetic energy to system capacity. Microgrids usually have *H* < 1 s compared to 2–10 s for large generation plants. Frequency stability is defined by the condition Δ*fpu* < Δ*f pu max* where allowable frequency deviation Δ*f pu max* in p.u. is typically 0.01–0.05 per unit. In [39] the authors derive the ODE ω*mech*Jdω*mech*/dt + ω<sup>2</sup> *mech*D = *Paccel* where ω*mech* is the mechanical speed of the generator in rad/s, J the moment of inertia (kg·m2), D a damping coefficient (Nm/s), and Paccel the power imbalance exerted on a generator rotor, to model the microgrid stability control problem. The NREL microgrid has a 300 kW Caterpillar diesel generator, whereas JBSA has no cogen. However, the same issues of stability and frequency control apply when there are no spinning resources. Some electric

codes are specifying ancillary control must be added to the EMS in order to handle ramp events of a certain magnitude and duration. See [40–42] for details.

#### *1.3. Climatology and Microgrid Architectures at the Three Locations*

As shown in Figure 5 the UTSA SkyImager has been deployed at 3 geographically diverse locations: Golden, Colorado on the rooftop of the ESIF building at NREL, in San Antonio, Texas at the CPS Energy microgrid facility at Joint Base San Antonio (JBSA) and the Engineering Building at UTSA, as well as in the Canary Islands, Spain at Tenerife and Caleta de Sebo. Each location presented unique challenges in terms of local climate, physical and cyber access, and microgrid design, equipment, operation, and customer needs.

**Figure 5.** SkyImager at (**a**) NREL, (**b**) JBSA, (**c**) Tenerife and (**d**) Caleta de Sebo.

The UTSA SkyImager was first conceived as a technology for providing accurate intra-hour irradiance forecasts as inputs to a microgrid management system that would then provide the utility with command and control of the microgrid in either connected or islanded mode. The Department of Energy INTEGRATE project [43] lasted for 18 months beginning on 6 March 2015 and partnered NREL, Omnetric-Siemens, CPS Energy, Duke Energy, and UTSA. The project goal was to increase the capacity of the electric grid to incorporate renewables by upgrading and optimizing architectures for control and communication in microgrids. There were three major components: (1) OpenFMB, a reference architecture that allows real time interaction among distributed intelligent nodes, (2) optimization with the Spectrum Power Microgrid Management System based on the Siemens SP7 Platform, and (3) PV and Load Forecasting using UTSA's applications for both intra-hour and day-ahead irradiance and building load forecasts. The OpenFMB framework leverages existing standards such as IEC's Common Information Model (CIM) semantic data model and the Internet of Things (IoT) publish/subscribe protocols (DDS, MQTT, and AMQP) to allow flexible integration of renewable energy and storage into the existing electric grid. The OpenFMB standard was ratified by the North American Energy Standards Board (NAESB) in March of 2016 and allows communication between diverse grid devices–meters, relays, inverters, capacitor bank controllers, etc. It allows federated message exchanges with readings such as kW, kVAR, V, I, frequency, phase, and State of Charge (SOC) published every 2 seconds as well as data-driven events, alarms, and control in near-real-time.

#### 1.3.1. SkyImager at National Renewable Energy Laboratory in Golden, CO

The site of the first SkyImager deployment was NREL in the Rocky Mountains. Golden's high elevation and mid-latitude interior continent geography results in a cool, dry climate. There are large seasonal and diurnal swings in temperature. At night, temperatures drop quickly and freezing temperatures are possible in some mountain locations year-round. The thin atmosphere allows for greater penetration of solar radiation. As a result of Colorado's distance from major sources of moisture (Pacific Ocean, Gulf of Mexico), precipitation is generally light in lower elevations.

Eastward-moving storms from the Pacific lose much of their moisture falling as rain or snow on the mountaintops. Eastern slopes receive relatively little rainfall, particularly in mid-winter. The SkyImager enclosure came equipped with a heater/fan that performed well at NREL. Given the

climate, it proved useful in keeping frost off the plastic dome. It adds to the expense and complexity of the technology and may not be required at other locations. Most installations of the security camera enclosure would be facing downward and perhaps under a building overhang. Used facing upward and exposed to the sky, there were issues with water getting inside the enclosure. A simple solution was silicon caulk applied at the base of the dome. In a typical security installation, a green tinted plastic dome is used with the enclosure to protect components from UV radiation. For all-sky imaging a clear plastic dome is a necessity. With any plastic material on a bright sunny day there can be issues with glare caused by the dome, but this was minor. The alternative is a glass dome but that has it own set of problems.

As shown in Figure 6, the microgrid at NREL was already well established and the process of deploying the SkyImager went relatively smoothly. Denver International Airport is located some 36 miles from Golden; this distance introduces some error in the Cloud Base Height for the ray-tracing algorithm originally used in the SkyImager. The ESIF building at NREL had the infrastructure necessary for easy installation of both the SkyImager and the Hardkernel Odroid C1 single board computer (SBC) used for load and day-ahead PV forecasts. Information was transferred using a Wi-Fi network on a LAN system. NREL also provided un-interruptible 120 VAC power, ample Ethernet connectivity, and excellent on-site weather and irradiance data. In addition to solar PV arrays, generation included a 500 kW wind power simulator and a 300 kW caterpillar diesel. A 300 kWh battery system provided energy storage and the load was separated into a controllable component (250 kW) and a critical load (250 kW).

**Figure 6.** Microgrid at NREL ESIF Building where the first SkyImager was deployed in 2015.

#### 1.3.2. SkyImager at San Antonio, TX, USA

Texas produces more electricity than any of the other 49 states, and as a result has its own interconnect ERCOT. In 2017, power statewide was generated by a variety of sources: natural gas (45%), coal (30%), wind (15%), and nuclear (9%). In 2014, wind replaced nuclear as the third-largest source of power and Texas now produces more wind power than any other state. Solar generation is increasing, but still relatively small for a state with abundant annual sunshine. Located in central Texas some 200 miles from the Gulf of Mexico, San Antonio is home to almost 1.5 million people and several military bases. CPS Energy serves San Antonio and is the nation's largest public power, natural gas and electric company. They are committed to renewables, funding a 400 MWac project with multiple PV plants (Alamo 1–7) close to San Antonio, and wind farms in West and South Texas. CPS Energy is among the top public power wind energy buyers in the nation and number one in Texas for solar

generation. In keeping with this commitment, TSERI was formed in 2001 as an alliance between CPS Energy and UTSA.

For San Antonio, the most significant local weather issue is low-level Gulf stratus [44]. Elevations of the terrain increase from sea level at the Gulf coast to almost 800 ft at San Antonio, and a moist air mass over the Gulf of Mexico will cool adiabatically to saturation as it moves upslope. Nocturnal radiational cooling causes cloud formation before midnight, resulting in a ceiling of 500–1000 feet. A solid cloud deck will cover much of central Texas and remain in place until late morning when the sun burns off the stratus and cumulus clouds begin to form. Forecasting Gulf stratus is an important problem for aviation; it is a matter of accurately predicting low-level wind flow (<5000 ft) with the most favored wind direction for stratus formation from 90◦ to 180◦. It is important to address these local weather conditions that occur below the spatial and temporal resolution of NWP, but are crucial for both inter-hour and day-ahead irradiance forecasts. Use of machine learning using local datasets and climatology will allow the information and intelligence of a study such as [44] to be incorporated in site-specific irradiance forecasts.

The Fort Sam Houston Library location at JBSA presented several unique challenges for the deployment of the UTSA hardware and software, challenges that provide valuable insights for other researchers. Many of the issues that arose were heavily dependent on the specific location. At JBSA, the Sky Imager was deployed using an edge-computing configuration with a wired Ethernet connection for cyber security. The JBSA microgrid is shown in Figure 7 and includes the Base Library building, solar arrays, inverters, and the pod housing the battery energy storage system (ESS). The need for accurate on-site meteorological observations necessitated installation of a complete MET Station atop a 10m antenna tower. A Campbell Scientific weatherproof instrument box at the tower base contained a National Instruments MyRio computer, a transformer, backup battery, and an Odroid C2 single board computer (SBC) for calculating the day-ahead load/PV forecasts. Atop the tower sat the SkyImager, a WXT520 Vaisala weather transmitter, and a pyranometer.

**Figure 7.** The Microgrid at Joint Base San Antonio, TX, USA.

In July of 2018 another SkyImager was deployed in San Antonio at the location of a university PV generation project. Funded by a DOE-SECO grant [45] in 2014, solar panels were installed on the Engineering Building, HEB University Center III, and Durango buildings at UTSA. In addition, equipment was installed to record measurements from 4 Combiners, 4 Inverters, 2 Kipp & Zonen CMP11 pyranometers, and a WXT520 Vaisala Weather Transmitter at the UCIII. Figure 8a displays the SkyImager and PV panels and Figure 8b shows combiners/inverters atop the Engineering Building. The only ingredient lacking to make this a research microgrid was energy storage.

**Figure 8.** (**a**) PV panels and SkyImager at UTSA; (**b**) Inverters & combiners, Engineering Bldg.
