**1. Introduction**

The United States electrical grids face a range of new challenges to safe and reliable operation: aging infrastructure, increased penetration of less predictable renewable generators to mitigate climate change and the increasing occurrence of extreme weather events all place stress on the grid [1]. To address these issues, more decentralized grid architectures have been proposed [2] based on distributed energy resources (DERs) and microgrids [3–5]. Collections of buildings with local DER and energy storage could operate in grid-disconnected (islanded) mode in case of outages, improving system resiliency [6,7]. Buildings that participate as DERs could also provide additional opportunities for energy storage using their thermal systems (Heating, Ventilation and Air Conditioning (HVAC) and Refrigeration) [8]. With the emergence of low-cost solar and battery storage, small-size microgrids are now a commercially viable option at sites with a high value for resilience and several products to

control them have appeared in the market [9–11]. However, microgrid software typically focuses on site protection and battery control and does not coordinate with control of building systems, especially HVAC [12].

HVAC systems represent the largest fraction of the energy use and demand in commercial buildings [13]. Traditional HVAC control strategies deploy rather simple control algorithms [14] that might reside in zone-level thermostats (in small and medium commercial buildings) or in a centralized building automation system (in larger commercial buildings). These algorithms rely on static schedules, irrespective of the actual occupancy of the building or grid needs. While more modern HVAC control strategies exist, they do not typically incorporate DER and are largely proprietary in their implementation, making them hard to extend.

The lack of interoperability between these new control systems limits the realization of the full potential for microgrid systems [15]. For instance, a microgrid controller that has very little information about the status of the building and its projected load cannot operate the battery optimally. Furthermore, there can be missed opportunities for HVAC and refrigeration controllers to take full advantage of periods of high solar generation to pre-cool or pre-heat, utilizing innate thermal storage in the mass of buildings and refrigerated goods.

To address this gap, this paper presents a software platform called "Solar+ Optimizer" (SPO), which was developed, deployed and tested in a pilot application at a fueling station with convenience store in Blue Lake, California, United States. The software provides optimal control of the building loads and DERs and has been built exclusively on open-source libraries. The controller takes into account the time-varying costs of energy and demand and the status of the grid connection to reduce the overall operational cost for the building owner, and it provides demand-response services to the grid. The platform is designed to be scalable, as well as vendor and protocol agnostic. This allows building managers to take advantage of a larger market of connected devices (both sensors and actuators) without being tied down to any particular manufacturer ecosystem, and it could enable less costly adaptation and modification of the system over the expected multi-decade lifetime of microgrid hardware.

The paper is organized as follows. Section 2 reviews the existing literature of advanced control studies with a focus on experimental and field studies. Section 3 introduces the overall software architecture and details the principal components within. Section 4 describes the deployment of the hardware and software on a case study and presents experimental results and analysis. Section 5 discusses the results and technical challenges encountered during the demonstration. The paper ends with conclusions and future work in Section 6.

#### **2. Literature Review and Contribution**

Research on advanced control strategies and algorithms (e.g., MPC and reinforcement learning) with application to building systems and DERs has grown significantly in the last decade [16], as the potential of these advanced controls to provide flexibility to the electrical grid has become more evident. Previous studies have investigated MPC applications in a range of HVAC and thermal energy storage systems in buildings: These include MPC utilizing an ice storage tank and building thermal mass [17,18]); MPC for Air Handlers (AHU) and Variable Air Volume (VAV) systems [19–21]; and MPC applied to window operation for mixed natural and mechanical ventilation in an office building [22,23]. Studies have also demonstrated the coordination of HVAC, energy storage and PV generation using MPC based controls in simulated commercial buildings [24–27]. However, most of these studies developed customized solutions, closely tied to the specific building and equipment setup; the gap that this work fills is an MPC application that is built to be extensible and scalable.

There are several control algorithms that have been used for optimizing building systems and DER. Linear programming was employed to minimize the conditional value at risk in the objective function while providing resilience and cost minimization in commercial buildings through local energy generation and storage [7,28]. Genetic algorithms have also been used to optimize the building thermal loads [29,30]. Reduced energy costs and improved occupant thermal comfort within buildings were achieved by using particle swarm optimization [31,32]. Dynamic programming was employed by Benjamin Heymann and Jiménez-Estévez [33] to reduce the energy costs while meeting the building load requirements. A neural network model was trained using the Levenberg–Marquardt algorithm to predict the optimal boiler operation period in commercial buildings [34]. Another emerging type of advanced control is Deep Reinforcement Learning (DRL) [35–40]. While DRL is a model-free approach, most DRL methods use detailed models to generate synthetic data for training purposes. EnergyPlus models of the buildings have been used during the training period [36–40], along with the Building Controls Virtual Test-Bed (BCVTB) [41], which has been leveraged for controls based on the co-simulation framework [37,38]. The DRL algorithms were used to minimize energy costs while maintaining thermal comfort by controlling room temperature setpoints [37], air flowrate in the VAV boxes [38], supply water temperature [39] or outdoor air damper positions [40]. This robust area of new research is promising, with many approaches to optimal control still in their infancy. There is not ye<sup>t</sup> an agreed-on standard approach for these problems.

While the majority of research on advanced control algorithms is conducted on simulations, some studies have deployed those advanced controls on real systems, which allows researchers to test their software in a live environment with unforeseen system behavior that is difficult or impossible to include in simulation. MPC control of multiple air handling units (AHUs) or rooftop units (RTUs) in multi-zone real buildings are demonstrated in [42–46]. Experiments using MPC to control building systems, behind-the-meter energy storage and DERs have also been demonstrated [47–49]. MPC based controllers were deployed in office buildings to determine the setpoints for the supply air temperature, fan speed and the zone air temperature for each AHU [42,43], and West et al. [42] additionally controlled the chilled water and hot water valve position. MPC has also been used to control the air conditioning in large spaces that were served by multiple RTUs by turning on/off different stages of operation of each RTU. These controls were demonstrated in a gym space of a university campus [44] and in a restaurant [45]. Carli et al. [46] used MPC to control the fan speed of a fan coil unit that supplies conditioned air to a single office space in a university building. Frequency regulation as a grid service (by varying the air flow rate setpoints, which resulted in modifying the fan speed) was implemented in the FLEXLAB -R testbed [50] at Lawrence Berkeley National Laboratory [47,48]. A public school building in southern Italy was chosen as a site of demonstration in [49], where the authors performed their optimizations in the cloud and send the control signals to the Internet of Things (IoT) devices and controllers that were retrofitted in the school building to control the battery and various loads through an intermediary gateway. DRL strategies have also been implemented in real buildings, although in small experiments. Chen et al. [51] used DRL to control the damper position of a VAV box in a single conference room in a building and Zhang and Lam [52] used it to control the supply water setpoint to the HVAC system in a experimental test-bed office in a university building.

A core barrier in progressing from simulations to real-world deployment of advanced controls is the lack of a robust and reliable software infrastructure to implement those controls in real-world building systems, which often have an eclectic mix of various controllers and systems in place. There has been work on middle-ware software platforms that collect data from various connected sensors and actuators across different systems within a building [42,43,46,48,49,53–56]. The ability to retrieve data from various IoT sensors and devices, store these data centrally and use them for simulations of better control algorithms have been demonstrated in [46,49,53–56]. Interfacing with the existing proprietary control software for gathering data and publishing control signals is another common solution, but this requires site specific implementations as seen in [42,43,48]. Bruno et al. [49] and Carli et al. [46] introduced a software stack and also demonstrate actual controls capabilities on real buildings based on decisions determined by the optimization engine. However, the architecture seems rather case specific with no mention of expansion capabilities to other types of buildings or systems. The VOLTTRON [57] platform has also been used in research studies for data collection and optimization of flexible building loads and grid integration, but these works have been in simulation [58–60], in laboratory environments [61], or are still in progress [62,63]. Most of the platforms reviewed here employ a publish-subscribe method of communication (usually Message Queue Telemetry Transport, MQTT [64]) for messaging, which relies upon a central broker, usually residing in the cloud. This is in line with broader trends for information technology and software services moving to a cloud-based software architecture, but the buildings industry is justifiably reluctant to adopt this model. Cybersecurity vulnerabilities and the possible loss of data and control signals during a network outage are major issues that concern building managers, particularly in commercial buildings. Furthermore, in a microgrid application there can be loss of network connectivity during a blackout, when the system is expected to continue working and provide resilience. Hence, having controls that are able to operate in a mode with local communication and computation may be required to enable widespread adoption of smart control platforms.

Among the field demonstrations of advanced controls, the scope of controls were limited to a very small subset of buildings, mostly office spaces and buildings on university campuses, laboratories or experimental test-beds. Experiments were conducted in well-instrumented environments and thus presented more measured data for advanced controls deployment than general buildings do. Most existing studies rely on proprietary software and do not describe the effort required to deploy both software and hardware, which is critical for replication. While some papers discuss software implementation, the focus was generally on communication and software architecture, with less emphasis on the integration of systems and controls with those software. Those that did demonstrate integration between the controller and the building, deployed a cloud based solution that is susceptible to network outages and more vulnerable to security risks. The SPO solution presented in this paper represents an advance in the field studies of advanced building controls by narrowing the gap between MPC based simulation studies and field demonstrations. The main contributions of this paper are:


## **3. Controller Architecture and Components**

The Solar+ Optimizer (SPO) is a software solution that has been developed to integrate sensors and controllers for building systems and DERs and to identify real-time optimal control actions for the connected systems such as building loads and batteries. It supports integration across multiple devices and protocols, as illustrated in the architecture diagram in Figure 1. Through support for several communication protocols and APIs, SPO allows integration of systems that are typical for small and medium commercial buildings. It can operate completely within a local network, but can also be configured to operate in tandem with cloud-based resources. This section describes the different software components of SPO that enable these capabilities.

**Figure 1.** Software architecture diagram of the Solar+ Optimizer system.

#### *3.1. eXtensible Building Operating System (XBOS)*

To coordinate the numerous heterogeneous connected devices and controllers within a building, robust network communication is a key requirement. Most commercial and academic solutions use middleware, i.e., software that resides between the hardware devices and other data sources that produce data and the applications that use these data. SPO uses XBOS, an open-source building operating system developed for real-time data acquisition from sensors and control of building actuators [65]. XBOS consists of the following components:

