*3.5. Load Profiling*

Standard load profiles [138], i.e., averaged models of customer energy consumption over time, have traditionally found their application in power grid capacity planning. However, standard load profiles are only accurate when considering many connected customers in conjunction and generally do not adequately reflect individual consumers' consumption characteristics. Smart meters can mitigate this situation and allow for capturing load profiles available in an unprecedented resolution. The enabled understanding of energy consumption profiles empowers users not only to better recognize how much energy they consume but also to compare their consumption profiles to the profiles of other dwellings [139]. This gives households greater control of their energy consumption and enables the adoption of more energy-efficient, and responsible behaviors [139–142]. Instead of considering the load profiles of buildings individually, it is often sufficient to know the *category* that better describes the dwelling. In other words, by categorizing the electrical power consumption, it is possible to approximate the load profile of a household sufficiently. It is therefore not unexpected that most of the existing load profiling techniques rely on clustering algorithms, such as *k*-means [143–145], fuzzy *k*-means [143], hierarchical clustering [143,146], Self-Organizing Maps (SOM) [143], neural networksbased clustering [147–149], Gaussian Mixture Models (GMM) [150,151], Density-Based Spatial Clustering (DBSCAN) [152], and agglomerative clustering [153]. Due to the high stochasticity and irregularity of household-level consumption, clustering techniques that analyze the variability and uncertainty of smart meter data have also been considered in the literature [150,151]. For example, the work by Lee et al. [143] proposes a two-stage (feature extraction and load pattern identification) *k*-means clustering for customers segmentation in residential demand response programs.

Load profiling results have been documented to find use in supporting and enhancing continuous energy audits in buildings that currently require multiple measurements [61,154]. Furthermore, the insights generated from load profiling can be used to enhance many other use cases of smart meter data. Eco-feedback techniques often utilize load profiles, e.g., to compare the consumption of individual days or different homes (see Section 3.1). For instance, in [141], an algorithm for computing the carbon footprint derived from load curves is presented. Likewise, load profiles can be used along with load demand forecasts (see Section 3.4) to generate optimal schedules of home appliance usage. This is presented in [155], in which a NILM-based energy managemen<sup>t</sup> system was developed to schedule controllable loads taking into account customers' preferences and overall satisfaction.

In summary, load profiling at the end-user level enables and potentiates consumercentric services. Furthermore, as load profiling can play an essential role in assisting the smart grid, it will become even more relevant to the individual consumer when Distributed Energy Resources (DERs) and local energy communities become ubiquitous and require the active participation of citizens [156].

#### **4. Open Research Challenges**

The range of customer-centric use cases for smart meter data contributes to numerous areas of daily living. Besides allowing for monetary savings, grid-friendly appliance scheduling, and the detection of atypical and anomalous appliance operations, it has been shown to serve as the enabling technology for AAL as well as the integration of RES, ESS, and EVs. Many of the underlying research challenges have been solved to a satisfactory degree to date, and corresponding commercial solutions are already on the market. During our survey of user-centric applications in Section 3, however, we identified obstacles to the enablement of the services, which potentially impact their widespread acceptance. We thus summarize the most important observed challenges as follows.

#### *4.1. Standardized Hardware and Data Formats*

As stated in Section 2, there are no universally acknowledged definitions of: (1) the parameters to be reported by smart meters; (2) the temporal resolutions at which they are

being made available; and (3) the interface using which service providers can access these data. As such, delivering the use cases to customers generally requires non-negligible adaptation efforts. A widespread approach to achieve compatibility nowadays is when the same company that rolls out and operates the smart meters also acts as the service provider. This "vendor lock-in", however, severely hampers the scale of services that can be provided, as well as their interoperability with other external services. Thus, creating an open ecosystem in which different stakeholders can synergistically combine their (often complementary) components to create an environment that leverages the full potential of smart meter data is currently not possible from a technological point of view. As one first step towards overcoming this obstacle, the International Electrotechnical Commission (IEC) has established a dedicated technical committee (*TC 85*) for the standardization of equipment, systems, and methods for the analysis of steady-state and transient electrical quantities. One of the committee's publications, *DIN IEC/TS 63297*, is a project report on "Sensing Devices for Non-Intrusive Load Monitoring" [157], seeking to unify the access to the required data from smart metering devices. Despite the ongoing standardization efforts, however, solutions that cater to the needs of metering and consumer-centric service providers alike remain to be found and widely adopted.

Besides the limited access to the electrical parameters measured by a smart meter, a second major impediment to the roll-out of services is the unavailability of a local execution environment for data processing code. This is particularly relevant to address privacy considerations, as detailed in Section 4.3. Most smart meters do not offer possibilities to run code apart from the device's (metering) firmware. While this is generally intentional, to prevent tampering with the reported data (e.g., to avoid electricity theft, cf. Section 3.2), it does not allow any of the user-centric services to be executed directly on the smart meter. As technology advances and embedded devices are gradually becoming more and more potent in processing power and the ability to run user code in dedicated sandboxes, the provision of an execution environment on smart metering devices appears as a promising and potentially groundbreaking approach. Retrofitting existing and coming smart meter generations will represent a challenge, given the expected operational life of smart meters (often more than a decade) and, more importantly, the expected evolution of software frameworks in the same period. One viable solution is to offload these services to dedicated data processing devices, such as local set-top boxes, edge-clouds, and Multi-access Edge Computing (MEC), or cloud computing in general. The widespread adoption of 5G networks (and more recent developments, such as 6G) will allow sending data of even finer temporal resolution to external processing devices. It can be expected that communication will no longer be the bottleneck, thus the execution of services can be distributed across the aforementioned range of possible processing devices in order to optimize consumerspecific expectations to reliability, privacy, and real-time needs. However, this requires standardized interfaces, (real-time) transport protocols, and data formats aligned with our above observations.

#### *4.2. Innovative Consumer-Centric Data Processing Algorithms*

Smart meter data analytics services are frequently based on the combination of data preprocessing with novel methods to find correlations, patterns, and outliers in the available input data. Although the concept of electrical load signature analysis has been investigated since 1985 [158], the underlying preprocessing methods (e.g., event detection and NILM) are still not perfect and yield mediocre disaggregation performances in certain settings [159]. Sometimes this limitation can be circumvented by enriching electrical data with other sensed parameters (e.g., ambient conditions) and combining the data collected from different dwellings. However, only when the full amount of information can be extracted from smart meter data, the complete spectrum of user-centric services can be realized. Increasing the data processing methods' reliability and accuracy is crucial for widespread user acceptance and remains a significant future research challenge.

However, the sole availability of a range of data processing services does not necessarily lead to their ubiquitous adoption. Rather, the selection of useful and necessary services is expected to differ significantly between users. Fitting all smart meters with the same processing methods will thus not only incur the excessive and unnecessary use of computational power but still not serve all customers' needs equally well. Ultimately, we expect customers to utilize services depending on their situations. This implies that they need to selectively decide which of the services are of relevance to them. Thus, helping users identify the required services and understand the privacy implications when sharing data with the service providers (cf. Section 4.3) is crucial. This is also well-aligned with Section 4.1, confirming that a more flexible configuration of services and corresponding data sources are needed.

Lastly, we would like to recall our statement from Section 2, in which we emphasize the enormous potential of analyzing *electrical* signals for the provision of user-centric services. In fact, smart meter data are primarily related to electrical quantities, for whose interpretation an in-depth understanding of electrical engineering and power engineering is required. Many of the use cases surveyed in Section 3 rely on the interpretation of time-series data (i.e., sequences of measurements), which, in turn, calls on the expertise of mathematicians and signal processing experts. Simultaneously, experts in artificial intelligence and machine learning (e.g., computer scientists) can contribute ye<sup>t</sup> different data analytics methods, especially when the volume of data to process is enormous. Smart meter data analytics is thus not only a cross-domain challenge, but also transdisciplinary research communities are inevitable to apply the state-of-the-art methods on smart meter data and thus enable accurate service provision. Finally, we expect that the same methods that apply to smart meter data can also find their application to other metered commodities, such as water or natural gas.

#### *4.3. User Privacy Protection*

The collection of smart meter data at high temporal resolutions bears the enormous potential to provide services to the electricity customers' benefit. Simultaneously, however, the appropriate protection of collected data against unauthorized third parties' access is strongly needed. The reason is straightforward: Any processing method applicable to captured data (cf. Section 3) can be equally well applied by an attacker, seeking to profile a building's inhabitants (see Section 3.5) or learn about their habits. Often, this includes learning about usual sequences of household activities from consumption profile (as discussed in Section 3.2). Solutions to ensure the secure transmission of smart meter data and adequate user privacy preservation are thus indispensable.

One method to circumvent security and privacy implications from the transmission of smart meter data to centralized processing systems (e.g., cloud computing) is their purely local processing. Due to the high resource requirements of many (pre-)processing methods and services (see, e.g., Section 2.2), however, this approach cannot always be applied. Particularly, when the data processing methods depend on parameters unavailable locally, corresponding computations must be executed on remote systems. Collaborative data processing approaches, i.e., the local extraction of features and their forwarding (devoid of most sensitive information) to remote data processing centers, represent an important future research direction. As a side effect, this also increases the services' adherence to the "data minimization" and "purpose limitation" requirements of data protection laws (such as the European Union's General Data Protection Regulation (GDPR) [160]).

When users cannot exert full control over the data their smart meters report, covering up characteristics in smart meter traces to hide user actions/intentions may also be necessary to protect their privacy. Current approaches mostly realize this functionality employing operating controllable generators or consumers to obfuscate the operation of sensitive appliances (e.g., [161,162]) or by intentionally falsifying reported data [27]. The potentially negative impact on the achievable services based on smart meter data, however,

needs to be weighed up individually by clients and their willingness to pay the "cost of privacy" [163].
