Global Quality of Service (QoX) Management for Wireless Networks

Abstract

In the fast-changing technological landscape, novel applications are emerging with the potential to reshape the world. These applications, while promising, impose stringent requirements in terms of quality of service (QoS). The advent of wireless networks like 5G, 6G and Wi-Fi 6 brings about resource management solutions to ensure these requirements while meeting the user expectations within the interconnected environment. Nevertheless, user behaviors are also evolving, highlighting the importance of satisfaction and quality of experience (QoE). Furthermore, changes in user behavior trigger shifts in business models, where the quality of business (QoBiz) takes on a pivotal role. This evolving ecosystem, encompassing QoS, QoE, and QoBiz, demands a comprehensive and adaptable approach that conventional QoS management frameworks fail to perform. This paper introduces an implementation methodology for a global QoS management model named QoXphere. The implementation methodology is grounded in machine learning techniques and addresses the multifaceted aspects of quality of service (QoX) and their interconnections within wireless networks. The objective is to facilitate dynamic resource management that not only elevates user satisfaction but also optimizes provider benefits. Real-world examples illustrate the methodology’s applicability in widely deployed networks, complemented by simulated scenarios of modern network environments that further validate the approach.

1. Introduction

In the rapidly evolving environment of technological innovations, novel applications and services [1] are emerging to revolutionize industries and redefine human interactions. New applications like self-driving cars, virtual reality, remote medical services, or factory automation [2] will soon become daily services. While these applications exhibit remarkable potential, they are also extremely demanding in their requirements, particularly in terms of ultra-low latency and ultra-high reliability [3,4].

The advent of new networks such as 5G, B5G, and 6G [5,6] has opened a realm of possibilities to address the emerging applications requirements. These modern networks not only bring the potential to meet the stringent needs of the new services but also introduce a more dynamic and responsive approach to handling network resources. However, to fully leverage the potential of these advanced networks, new quality of service (QoS) [7] frameworks are required. The International Telecommunication Union—Telecommunication Standardization Sector (ITU-T) emphasizes in Recommendation Y.3106 [8] the need to develop more granular QoS control mechanisms to fully exploit the capabilities of these networks. Furthermore, this recommendation highlights the importance of enabling user-initiated QoS management to support the varied performance demands of different user profiles.

As we move into the era of modern networks, the way users interact with technology continues to evolve, shaping new patterns of behavior that align with the changing demands of a digitally connected scenario. This transformative momentum has notably gained pace in recent years, fueled by factors like the global COVID-19 pandemic. Consequently, as users seek more immersive and interconnected experiences, the significance of user satisfaction with the services—intricately connected to their quality of experience (QoE) [9,10,11]—becomes a pivotal factor [12,13]. Therefore, it is crucial to support user-based QoS control mechanisms to accommodate diverse service performance needs. This approach ensures optimal QoE by complying with a service level specification (SLS) established through negotiations between a customer and a service provider in a service level agreement (SLA) [14].

Regarding SLAs, it is noteworthy that the ongoing technological evolution, along with changes in user behavior, will have a profound impact on economic and business models. As advanced networks continue to develop, new possibilities and opportunities will arise for businesses, service providers, and industries to innovate and redefine their offerings. As a result, the telecommunications market is expected to undergo significant changes, leading to new revenue opportunities, disruptive business models, and more flexible market dynamics. Thus, organizations will need to quickly adapt to harness the potential of these advanced networks to meet user requirements. In this complex environment, quality of business (QoBiz) [15] also becomes crucial.

In light of all the previously mentioned trends, in this ever-changing digital era, new models for the dynamic management of quality of service (QoS), quality of experience (QoE), and quality of business (QoBiz) are imperative. Traditional approaches to network management no longer suffice in capturing the intricate interplay between all these dimensions of the quality of service, named lately as QoX [16,17,18]. Hence, QoX is a comprehensive concept combining various quality dimensions, including QoS, QoE, QoBiz, and other indicators, to provide a holistic approach to what a global quality of service management system should handle. As user expectations evolve with modern networks, services become increasingly sophisticated. A new QoX framework is needed to ensure not only an optimal service performance (QoS) but also a seamless and enjoyable user experience (QoE), all while aligning with overarching business objectives (QoBiz).

In this paper, a novel implementation methodology for a global QoS management model named QoXphere [19] is proposed. Since the QoXphere model was defined before modern networks emerged, adapting it to current technological advancements, evolving user behaviors, and market shifts requires innovative methods and strategies. The proposed approach leverages artificial intelligence and machine learning to integrate all the required mechanisms for effective QoX management in modern networks.

The remainder of the paper is organized as follows: Section 2 provides context for the challenge and summarizes the background information. Section 3 details the updates to the QoX model and the proposed methodology for implementation in modern wireless networks. Section 4 presents and analyzes assessment outcomes using real-world examples, illustrating the methodology’s applicability in widely deployed networks with comprehensive explanations. Finally, Section 5 offers conclusions and final remarks.

2. Background

Over time, significant scientific and standardization efforts have been devoted to defining models and frameworks for QoS management. This section will review the evolution of these models, introducing basic concepts and detailing the foundational theories and methodologies that have shaped current QoS management practices. Understanding the progress and advancements in these frameworks highlights the need for comprehensive models, such as the QoXphere model, that address all the QoS aspects (QoX). The review also underscores the necessity to update the QoXphere model to ensure its effective implementation in modern networks. This task presents a significant challenge due to the complex interplay between multiple QoX aspects and quality indicators, making it difficult to develop a suitable methodology for such implementation. Leveraging advanced techniques, such as machine learning, will be crucial for overcoming this challenge and achieving successful deployment. Therefore, this section will conclude with a review of the most commonly used machine learning techniques in QoS management, specifically tailored to the requirements of the QoXphere model.

2.1. Contextualizing the Challenge

To comprehensively address the need for integrating all the dimensions of the quality of service (QoX) [16,17,18], it is important to consider the foundational definition of QoS. Traditionally defined as “the collective effect of service performance which determines the degree of satisfaction of a user of the service” [7], QoS was initially focused on the operational quality of the network (network performance-NP). However, as the technological landscape has evolved, new metrics emphasizing user experience (QoE) and market dynamics (QoBiz) have emerged. As a result, models and frameworks for QoS management have advanced to encompass this broader evolving approach to global quality of service (QoX).

This evolution is depicted in Figure 1, where Moorsel [20] illustrates the quality of service concerns among various participants in an Internet ecosystem. Each participant, or “player,” interacts with both service providers and customers (shown at the bottom and top of Figure 1, respectively). The figure underscores the need to adapt and integrate diverse dimensions of QoS to meet the changing demands and expectations of all stakeholders involved.

Services are received from providers and offered to customers. Each player has specific metrics of interest: Internet service providers focus on bandwidth, application service providers focus on the number of supported customers, and end customers focus on perceived response time. The QoE box in Figure 1 shows the level of quality of experience (QoE) each player receives from the service provider. Additionally, the player provides a service to customers using its own resources and the received service, resulting in a specific QoS (shown in the QoS box). This service is then delivered to the customers.

The right-hand side of Figure 1 illustrates the financial flow related to quality of business (QoBiz) metrics. QoBiz is affected by both costs and revenues. Costs include the expense of resources needed to achieve a specific QoS level and payments to service providers for delivering a certain QoE. High QoE can increase a participant’s willingness to pay, linking QoE and QoBiz. Revenue comes from customers, impacting the QoBiz metrics. Evaluating QoBiz requires considering these cost and revenue factors.

Similar to Moorsel, most scientific literature and standards recognize that QoS and QoE, while distinct, are interrelated concepts with overlapping connections [21]. Effective management of these relationships requires adept formulation of the connections between the key performance indicators (KPIs) [22] and the key quality indicators (KQIs) [23] to meet user requirements [8,24,25,26,27,28,29,30,31,32,33]. Aligning QoE with service level agreements (SLAs) [14,32] guarantees that service providers fulfill their commitments (QoBiz), fostering trust and reliability among customers [18]. A positive QoE not only reduces churn rates but also leads to increased customer fidelity, driving growth through word-of-mouth referrals. Moreover, optimizing QoE and SLAs to meet the service level objectives (SLOs) [21,34] can directly influence the key business objectives (KBOs) [11,15], such as customer lifetime value, revenue generation, and profitability.

Kilkki’s Figure 2, in “Quality of Experience in Communications Ecosystem” [21], adeptly illustrates these interrelations. It emphasizes that while customers may express a desire for high QoS, their actions, like purchasing or switching providers, reflect their actual experiences and satisfaction. Misalignment between customer expectations and technical implementations can lead to discrepancies, highlighting the complexity of integrating QoS into both business and technology domains. Figure 2 presents the foundational terminology needed to navigate this intricate interface.

By recognizing the integral link between QoS, QoE, and QoBiz—essentially addressing QoX—organizations can cultivate a customer-centric approach, fueling long-term success and competitive advantage in the dynamic telecommunications market landscape.

2.2. QoX Management Models

Over the past decades, many efforts have been placed into developing global QoS frameworks and models [23,32,33,35,36,37,38,39,40,41,42]. Defining the interrelationships of all the involved service quality indicators of QoS/QoE/QoBiz (QoX) for enabling dynamic and global QoX management has undoubtedly been the major challenge in shaping these models over time.

In this particular context, based on the guidelines established by standardization bodies [7,14,15,22,23,32,33,37,38,39,40,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60], our research team developed a model named QoXphere [19], specifically designed for the dynamic and global QoX management.

The QoXphere model (Figure 3) was built upon a user-centered and business-driven QoS design framework [14,39,61]. Characterized by an innovative spherical, iterative, adaptable, and multi-layered architecture (Figure 4), it aimed to embrace the diverse quality of service dimensions (NP, QoS, QoE, QoBiz) while upholding the fundamental and basic quality of service standards. One of the key priorities in the design of the QoXphere model was to ensure its adaptability and evolution in line with the future networks, technologies, user requirements, and telecommunication market. In this regard, its architecture (Figure 5) was founded upon the management of indicators that are both network-agnostic, technology-agnostic, and user profile-agnostic.

Accordingly, the lowest layer of the QoXphere encompasses the most technical, objective, and intrinsic aspect of quality of service, which was the sole consideration of QoS management when this field of study commenced and is now referred to as network performance (NP). Its definition takes into account the most important recommendations and standards related to network quality and resource management [54,59,60,62,63,64].

The subsequent level embraces the QoS framework established in the ITU-T’s Recommendation G.1000 [39], which is still in force to this day. This framework is mandatory for any QoS management model striving to be effective, as it aligns the perspectives of the user, service provider, and regulator.

The next layer focuses on user experience (QoE) and satisfaction, considering the foundational principles outlined in the basic QoE management standards [10,11,65].

Finally, but certainly not least, the highest stratum of the QoXphere model encompasses all facets concerning business quality (QoBiz). Built upon models such as eTOM [58,61,66,67,68] and business objectives, it establishes a direct connection with user satisfaction to foster loyalty and, consequently, provider profitability, all within the regulatory framework.

As noted earlier, a key priority in developing the QoXphere model was to ensure it could adapt to rapidly changing technologies and evolving user and market needs. The arrival of advanced networks (5G, B5G, and 6G) together with the widespread adoption of novel applications and services (delivered through cloud computing, virtualization, and SDN technologies) brings about such a profound shift in network management, user behaviors and market trends that adaptation becomes essential for all the previously defined QoS/QoE/QoBiz models, even QoXphere.

In response to this imperative need for adaptation, and in order to ensure seamless QoS and user experience within the dynamic interconnected environment, new QoS standards [8,11,24,25,26,27,28,29,30,31,34,69,70,71,72,73,74,75,76] and scientific research [12,42,77,78,79,80,81,82,83,84,85,86,87] have emerged. Hence, new guidelines and recommendations have been released to support the definition and adaptation of global quality of service management models to the changing characteristics of modern networks.

As an example, the ITU-T’s Y.3160 [34] recommends considering the key service level objectives (SLOs) and their corresponding values for configuring and designing the communication service (Figure 6a).

SLO-oriented acceptance specifically pertains to the evaluation and assurance process aimed at determining whether the service provided to users aligns with the SLA and meets their requirements. This process should encompass the service function and the KQI/KPI identification, as shown in Figure 6a, and it must be taken into account that different services, network technologies, contexts, and user profiles may have different requirements and quality indicators (Figure 6b).

In this regard, many authors have remarked on the importance of considering the different influence factors (IFs) [88,89,90,91,92,93] for the QoX assessment considering the factors influencing the quality of experience (QoE) can be categorized into three primary groups: system influence factors, context influence factors, and human influence. Certain authors [42] propose that business aspects should be treated as a distinct group of QoE influence factors. Conversely, there are those [20,94,95] who argue that these aspects should be regarded as an additional dimension of the overall service quality, such as within the realms of QoE or QoS (see Figure 1).

Nevertheless, even with this increased understanding of the relationships of the QoS/QoE/QoBiz, it remains imperative to possess mechanisms that can guarantee the service level objectives, ensuring user satisfaction through QoX management, independently of user profiles/requirements, network environments/technologies, and types of applications/services.

In this context, the integration of artificial intelligence and machine learning (ML) mechanisms has risen to the forefront as an indispensable enabler. In today’s digital age, the traditional, deterministic approaches to managing quality of service (QoS), quality of experience (QoE), and quality of business (QoBiz) often fall short in addressing the dynamic and multifaceted nature of the modern network ecosystem. Machine learning offers adaptive capabilities necessary to navigate the complexities of such diverse scenarios and ensure optimal SLOs and QoE across the board. Several compelling reasons support the integration of machine learning (ML) mechanisms in this context:

• Diverse user profiles and requirements: Modern communication networks serve a vast array of users with varying profiles, preferences, and demands. Machine learning can tailor services to meet the specific needs of each user, thereby enhancing their experience, by being trained on historical data and adapting to individual behavior.
• Heterogeneous network environments and technologies: With the coexistence of numerous network technologies, ranging from traditional wired or Wi-Fi networks to cutting-edge 5G and beyond, together with other network environments using network functions virtualization (NFV), software-defined networking (SDN), and network slicing, maintaining consistent QoE can be challenging. ML models excel at adapting to different network conditions and optimizing performance accordingly.
• Multitude of applications and services: The proliferation of applications and services, from video streaming to IoT (Internet of Things) applications, brings a multitude of quality requirements. ML’s ability to analyze data in real time allows for dynamic optimization and allocation of resources to ensure seamless user experiences across this diverse application landscape.
• Dynamic nature of user behavior: User behavior can change rapidly. What satisfies a user one moment may not be sufficient the next. Machine learning excels in recognizing patterns in user behavior and adapting QoE management strategies accordingly.
• Proactive issue mitigation: Machine learning models can detect anomalies and emerging issues early, allowing for proactive mitigation and rapid response to potential disruptions in service quality. This proactive approach enhances user satisfaction by preventing or minimizing interruptions.
• Resource optimization: Effective QoE management also involves resource allocation and machine learning can optimize resource utilization by adapting to network conditions, application demands, and user requirements, thereby ensuring efficient service delivery.
• Real-time decision-making: Machine learning systems make real-time decisions based on the analysis of vast datasets. This capability is indispensable for meeting service level objectives (SLOs) of the QoBiz and QoE expectations in dynamic and often unpredictable environments.

In essence, machine learning offers a data-driven, adaptive, and dynamic approach to QoX management that is essential in the face of the complex and ever-changing digital landscape. By harnessing the power of ML, organizations can stay ahead of the curve, continuously improving their QoX offerings and addressing the diverse needs of users, regardless of their profiles, the technologies they employ, or the applications they rely on.

2.3. Machine Learning Methodologies for QoS/QoE/QoBiz (QoX) Management

A revision of the most recent standards and scientific contributions in this area will help to understand the potential of machine learning to revolutionize traditional QoS management approaches. Through this exploration, the aim is to elucidate how our proposed QoX management methodology has been designed to achieve enhanced QoX management, regardless of the complexities posed by user preferences, network dynamics, and the intricacies of modern network applications and services.

Many standardization bodies have introduced machine learning-based models and frameworks in recent years for comprehensive QoS/QoE management. Particularly noteworthy is the exceptional effort undertaken by the Focus Group on Machine Learning for Future Networks, including 5G, established by the ITU-T’s Study Group 13 in 2017 [96]. Since its founding, this group has developed remarkably impactful recommendations in the realm of modern network environments, with a specific emphasis on QoS/QoE [8,31,75,97,98].

The functional model of machine learning-based QoS assurance defined by this group, presented in Y.3170 [97], describes a practical model for ensuring QoS through machine learning (Figure 7).

The machine learning engine within this model incorporates functional components to support diverse performance requirements for heterogeneous networks and services. The functionalities of each of these components are well detailed in this recommendation defining comprehensive prerequisites and operational specifications for QoS assurance driven by machine learning in the context of modern networks (referred to as IMT-2020 within ITU). Basically, the primary functions of these components are:

• QoS Data Collection: This component gathers QoS-related raw data from User Equipment (UE), Access Networks (AN), Core Networks (CN), and QoS key performance indicators (KPIs).
• QoS Data Pre-processing: The pre-processing component cleans the collected IMT-2020 QoS raw data by removing noise and transforming it into a unified format. It also updates the transformed data into the historical QoS data repository.
• Modeling and Training: Given that an anomaly is a pattern in the data that deviates from expected behavior (refer to clause 3.1.1), training machine learning models based on historical QoS data is essential. These models are used to detect and predict QoS anomalies. Initial detection and prediction are carried out using predefined models.
• QoS Anomaly Detection and Prediction: The machine learning models trained on historical data enable the detection and prediction of QoS anomalies. This process ensures that deviations from expected performance are identified promptly.
• QoS Decision-Making: Based on the results of anomaly detection and prediction, the decision-making component formulates the modern network’s QoS decisions. It then sends enforcement policies to the user plane to implement the QoS policies and reports the decisions to the management plane for comprehensive QoS assurance management.

To effectively assure QoS using machine learning, the system must meet high-level requirements for each functional component. These requirements ensure that the data collected are accurate and comprehensive, the pre-processing is thorough and efficient, the models are trained on robust historical data, and the decision-making process is informed and timely.

Table 1. Functional requirements for machine learning-based QoS assurance of ITU-T Rec. Y.3170.

	Functional Requirements
QoS data collection	Required: Collecting both static and dynamic QoS data along with QoS key performance indicators (KPIs) is mandatory from user equipment (UE), access network (AN), and core network (CN).
QoS data pre-processing	Required: Data format of the collected QoS raw data be extracted and transformed into understandable, unified and easy-to-use structures. Required: Noisy QoS data to be cleaned and filtered. Recommended: QoS pre-processed data to be normalized and unified.
QoS data repository	Required: QoS pre-processed data to be stored. Recommended: QoS-related anomalies and the corresponding collected IMT 2020 QoS pre-processed data to be labeled, if applicable.
Modeling and training	Required: Machine learning models be constructed based on the pre-processed QoS data and QoS KPI parameters (e.g., machine learning models: supervised learning, unsupervised learning, semi-supervised learning, deep learning, reinforcement learning, either alone or in combination). Recommended: Machine learning models to be trained based on the available pre-processed QoS data and QoS KPIs.
QoS/QoE correlation	Recommended: Besides the technical factors, various non-technical factors exist that may influence user QoE, e.g., device type, user emotion, habit, and expectation. It is useful to create an individual profile for each user that includes their preferences, habits, and interests. A user does not usually like to spend much time answering questions to create a profile model. As an alternative, a user profile can be built using machine learning-based QoS/QoE correlation. Recommended: Machine learning-based models and correlations between pre-processed QoS data and user QoE to be constructed. Recommended: Machine learning models to be trained for the correlations between pre-processed QoS data and user QoE.
QoS anomaly	Required: Detection of QoS-related anomalies to be supported based on machine learning models. Required: Prediction of QoS-related anomalies to be supported based on machine learning models. Required: QoS anomaly root cause detection to be supported based on the machine learning models. Required: Detected QoS anomalies be stored in the history QoS data repository.
QoS policy decision	Required: Enforcement of QoS decision policies to be supported on the user plane. Required: Reporting of QoS decision policies to be supported to the management plane for resource re-scheduling, network optimization, and planning.

offers a summary of the most crucial functional requirements outlined in the ITU-T’s Recommendation Y.3170.

In addition to these contributions, the ITU-T has engaged in revising and updating relevant recommendations [11,72,73,76,99] to cater to the evolving landscape. Other standardization bodies, such as ETSI, 3GPP [100], or 5G-PPP [101], are also actively contributing to the definition of models and frameworks related to ML QoS management in modern network scenarios.

Beyond the contributions made by standardization bodies, considerable scientific research has also been conducted recently [89,102,103,104,105,106,107] considering the integration of machine learning for providing QoS/QoE. Whereas certain authors embrace the ITU-T’s Recommendation Y.3170 as the foundation for their approach [108], others enhance their work by integrating business considerations into their proposals [109].

In a broader context, it is widely acknowledged among researchers that machine learning can bring advantages when addressing the following essential aspects crucial for global quality of service (QoX) management:

• Comprehensive user and context classification: Significant key quality indicators (KQIs) for the distinct user profiles/contexts [110], services [111], technologies [112], and business environment [113] should be inferred. Accurate classification of user profiles and environments for each service/technology requires a thorough consideration of the influence factors (FI) [77,89,90] for each unique environment (Figure 8), and machine learning techniques can assist in this challenging endeavor.
• Collecting, pre-processing, modeling and training appropriate QoS data: Crucial KPIs should be identified [114] to be collected and analyzed [115] based on the essential KQIs for each environment. Machine learning mechanisms, capable of smartly adapting to the network environment and responding to anomalies and dynamic circumstances, can also find application in this field [116].
• QoE/QoE correlation: The ability to model quality of experience (QoE) based on quality of service (QoS) is of paramount importance. Machine learning has frequently been proposed to undertake this pivotal task, which can alleviate the laborious process of conducting surveys [84,85,90,106,117].
• User satisfaction and churn avoidance: Estimating user satisfaction through QoE assessment is crucial for preventing churn [109,113,118,119,120] while ensuring service level objectives (SLOs) [121].
• Continuously align SLAs with user expectations: This alignment should always be guided by the overarching business objectives and the user expectations [79,121] and should constitute an iterative process, guaranteeing user satisfaction and provider profitability.

In spite of all these advancements and proposals, it becomes imperative to establish a comprehensive global framework for managing global quality of service (QoX), embracing, harmonizing, and synchronizing the various perspectives discussed in previous paragraphs. Hence, this framework could serve as an umbrella for all types of users, contexts, services, and technological environments.

In the near future, it is envisaged that widely deployed network technologies, such as Wi-Fi (802.11), collaborate with less established and forthcoming technologies (such as 5G, B5G, or 6G) with the aim of optimizing all the dimensions encompassed by QoX (QoS/QoE/QoBiz). Therefore, in addition to novel QoS models and frameworks, adaptive machine learning methodologies are also required to deploy these QoS models in all variations of converging ecosystems within the future network landscapes.

In the following sections, we will introduce our comprehensive approach to effectively tackling this challenge through our proposal.

3. Materials and Methods

Building upon the comprehensive analysis of the intricate QoX, as discussed in the previous section, we will now introduce our evolving QoX model and the proposed implementing methodology for modern networks.

3.1. QoXphere Model

The QoXphere model [19] was designed to make it capable of evolving and adapting to changes in networks, technologies, and, of course, the shifting requirements of users and the telecommunications market. Consequently, to address this advancing landscape, our model has been adapted to accommodate this diversity and the latest research and standards.

The architecture of the updated QoXphere is shown in Figure 9. We have incorporated into the figure the framework established by the ITU-T’s Y.3170 [92] (three different planes) to show how it aligns seamlessly with the recently established standards.

Next, the updates implemented in each layer of QoXphere are detailed along with the rationale behind them, whether it is to align with the new standards or to tailor it for modern network environments:

• Intrinsic QoS: The lowest layer of the QoXphere, as mentioned before, refers to the most technical, objective, and intrinsic aspect of quality of service. When the model was designed, there was not as much diversity in services and technologies, and the performance indicators were similar in all cases. In today’s dynamic environment, where a multitude of services and technologies coexist, the concept of intrinsic QoS now necessitates a more adaptable approach. Nowadays, the requirements to ensure a specific service class can vary significantly from one service to another [24,31]. Thus, this layer acknowledges that key performance indicators (KPIs) need to be customizable and responsive to the unique requirements of each service and user profile, reflecting the contemporary complexity of network demands. As a result, the intrinsic QoS layer has transformed into a dynamic framework that can seamlessly integrate a spectrum of key performance metrics and parameters (KPPs). This adaptability ensures that the model can effectively handle the intricate interactions among an array of services, users and technologies, catering to the ever-changing landscape of modern networks.
• Quality of Service (ITU-T Recommendation G.1000 framework): This layer was defined to accommodate the ITU-T’s G.1000 Recommendation QoS framework [39], which is still in force today. This framework was designed to align the perspectives of the three key stakeholders involved in QoS management: the user, the service provider, and the regulator. It acknowledges that different stakeholders have varying priorities and concerns regarding QoS achievement. By aligning these perspectives, the framework helps create a common understanding of what constitutes acceptable quality levels and how they should be measured and managed. It is worth noting that while the principles outlined in the ITU-T’S G.1000 Rec. provide a strong foundation for QoS management, it is essential to recognize that the field of telecommunications and network management has evolved significantly since the recommendation was first established. Modern networks present new challenges and opportunities that may require updating terminologies and methodologies of the G.1000 to remain effective, something that is already being addressed by the ITU-T’s Study Group 12 (SG-12). In particular, the technical report “Roadmap for QoS and QoE in the ITU-T’s Study Group 12 Context” (TR-RQ) [122] was approved in the last meeting of this study group, celebrated in September 2023. This report presents a collection of QoS/QoE terms and related definitions, established by THE ITU-T’S recommendations over time, and suggests eliminating overlapping concepts and clarifying the meaning, differences, relationship, and appropriate use for each term. Thus, it was agreed that the term QoP could be replaced with QoE, and we have adopted the same approach in our model in line with the emerging trends and comprehensive representation of the user experience. Therefore, this layer now serves also as a passage connecting QoS and QoE. Furthermore, within the TR-RQ report [122], several conceptual models are discussed that encompass various aspects contributing to the composition of quality of experience (QoE). These models are built upon different influencing factors (IFs) that impact both in QoS and QoE assessment. Most of the revised models, as stated before, proposed considering the system, human, and context factors [42,93,123,124], but some of them also take into consideration business factors, such as the proposed model of the TR-RQ (Figure 10).

As academic members of the ITU-T, we actively participate in new contributions and insights, and in this regard, we also made some proposals in this report [125] suggesting the use of machine learning in the QoS/QoE-related recommendation to update.

• Quality of Experience (QoE): In earlier versions of our model, this layer encompassed the quality experienced by the users. However, since this dimension has been chosen to be shifted to the lower level as a replacement for QoP, it has been decided to establish this layer as the link between QoE and QoBiz through the connection of user satisfaction and churn. Accordingly, we have updated the name of this layer to “Assessed QoS”, given that it already provides an indication of whether quality objectives are being achieved or not through the key risk indicators (KRIs).
• Quality of business (QoBiz): The upper layer of the QoXphere model encompasses all the aspects concerning business quality (QoBiz). As stated before, the telecommunications market is undergoing substantial changes with the emergence of novel applications and services, along with a profound transformation in user behavior. In this regard, there has been a need to update this layer as well to make it more adaptable to the distinct services and market segments. New elements, such as the service level objectives (SLOs), have been considered and defined through the key business objectives (KBOs) to accomplish this goal. The ITU-T’s Rec. Y.3160 [34] designates the SLO as the fundamental service level objectives specification, defined in terms of key business objectives (KBOs) and associated metrics, thresholds, and tolerances. Thus, after defining the SLA based on user requirements and expectations (KQIs), service providers can translate them into SLO attributes, serving as a link with the lower layer through the SLA/SLO relationship. This linkage completes the cycle of effective QoX management pursued with the model, enabling an iterative process that should lead to the continuous improvement of QoS, QoE and QoBiz. The SLOs can directly be adapted (Operational Efficiency-Op-Eff) within the resulting values for the KQIs, KPIs, KRIs, and KBOs in each of the model iterations.

As previously mentioned, the QoXphere model has evolved to be adaptable to any environment within the modern network ecosystem. Nonetheless, an application methodology is necessary to implement the model across the diverse array of scenarios found within modern networks. The following subsection outlines our proposal for this methodology.

3.2. QoXphere Model Implementation Methodology

The proposed methodology aims to aid in the implementation of the QoXphere model within modern real-world network scenarios. The primary challenge in this regard is to establish the intricate relationships and dependencies among the various QoS dimensions incorporated into the model. Therefore, the main objective is to dynamically align these components in accordance with the ITU-T’s Rec. G.1000 guidelines to effectively meet user requirements, preserve their satisfaction, reduce churn, and thereby ensure provider benefits. To achieve this goal, a methodology that combines machine learning techniques according to the ITU-T’s Rec. Y.3170 functional model for ML-based QoS assurance is proposed (Figure 11).

As stated in the ITU-T’s G.1000 Rec. [39], for any QoS management to be successful, the identification of the key quality indicators (KQIs) relevant to the users is crucial. Despite appearing straightforward, this process can evolve into a considerably intricate endeavor, especially within the context of modern networks. In such scenarios, network responses can be significantly shaped by user behavior and a myriad of contextual and non-contextual elements that are beyond the control of providers.

Therefore, a preliminary analysis becomes crucial to establish influence factors associated with each specific scenario and user profile. In being aware of the importance of this initial step, the approach integrates both contextual and non-contextual information collected through extensive big data analysis (step #1 in Figure 11). Unsupervised machine learning techniques (specifically clustering analysis) are suggested to infer distinct scenarios and user profiles from the network-recorded data (context extraction in Figure 11). This data will enable the acquisition of the influence factors related to the user’s context (such as location, scenario type, day/time/usage duration, etc.), as well as the so-called system-related influence factors (device type, operating system, etc.).

On the other hand, it is proposed to capture the influence factors that could not be extracted from the network, namely those categorized as human factors (such as age, gender, experience, emotions, requirements, expectations, etc.), through surveys (step #2 in Figure 11).

Once all the influencing factors are established, taking into account the survey results regarding expectations and requirements, inductive supervised learning is suggested. This approach uses the survey data as a training set to infer the rules that will allow us to identify the relevant key quality indicators (KQIs) for each type of user in each scenario (step #3 in Figure 11). Inductive supervised learning will eliminate the need for the laborious survey process, except for the initial training stage.

Once the KQIs are determined, key performance indicators (KPIs) and their associated key performance parameters (KPPs) will be identified in accordance with established regulations [23,33]. Suitable measurement systems will be defined to carry out measurements related to network performance according to the defined KPPs/KPIs (step #4 in Figure 11). This is the starting point of the control plane’s function with the collection of network performance (NP) data, as described in the reference framework [97], which is reflected in our QoXphere model as the “intrinsic QoS” layer (step #5 in Figure 11).

Based on the QoS data associated with the intrinsic QoS layer, it is recommended to employ unsupervised ML techniques to detect and predict anomalies, as outlined in the reference framework (Figure 7). The result of this analysis constitutes the first point of intervention where corrective actions can be implemented to improve quality of service (QoS).

Within the user plane of the ITU-T’s Recommendation Y.3170, the two intermediary layers of our QoXphere (QoS and Assessed QoS in Figure 9) are located. The method proposed for QoX management in this plane also follows the guidelines outlined in Recommendation Y.3170. Automated mapping of NP with QoE is considered fundamental. For automating this correlation, the use of supervised machine learning algorithms (regression models, etc.) is suggested [84,85,90,106,117]. In this case, it is essential to employ both objective data (network quality) and subjective data related to user experience and satisfaction collected through surveys (step #6 and NP/QoE correlation model in Figure 11). Contextual and non-contextual influencing factors will also be crucial when analyzing QoE and user satisfaction. For this reason, once again, they are included in the training set to learn the rules that will provide the predicted QoE in Figure 11.

Machine learning will obviate the necessity of iteratively conducting surveys to capture QoE each time, except for the initial survey that trains the system. This training dataset will be used to derive the rules that govern the NP/QoE correlation model.

Based on QoE results, the satisfaction model (CSAT) [126] will estimate user satisfaction with the service. This constitutes another important point of intervention where corrective actions based on detected Key Risk Indicators (KRIs) may be necessary. These KRIs could lead to customer churn and potentially affect the business model.

Finally, the decision for the QoS policy is made within the management plane, as considered in the ITU-T’s Y.3170 Rec. framework. Based on the outcomes of anomaly detection/prediction in the network and QoE results, the key business objectives (KBOs) will be updated (step #7 in Figure 11). KBOs are derived from the SLO defined through business areas deemed crucial for each company and must be aligned through operational efficiency to boost revenue, curtail costs, and enhance customer experience. Billing, advertising, QoS requirement adjustments, and other additional measures should be examined to update the SLA based on the results obtained from the user and control planes (step #8 in Figure 11). This linkage completes the cycle of effective QoX management, enabling an iterative process that should lead to the continuous improvement of QoS, QoE and QoBiz.

4. Results

In order to validate the proposed methodology, preliminary case studies have been conducted in real-world scenarios with widely deployed technologies. The results of these studies have been complemented by others from simulated modern network environments that further validate the approach.

For the real-world scenarios, the university campus environment has been selected. The decision was made to carry out these experiments using Wi-Fi technology, as it is widely used in these environments due to the availability and reliable performance of the eduroam academic network (education roaming). Eduroam’s widespread use in worldwide academic and scientific institutions ensures high levels of accessibility and participation in experiments, as most university and research users use it regularly. Moreover, utilizing Wi-Fi in a university setting allows the replication of everyday user situations, enhancing the validity of the results since users may use this network for both academic and personal activities, providing a more realistic reflection of the environment. A thorough validation of the methodology requires collecting pertinent data and rigorously analyzing the results. Furthermore, using our university campus environment simplifies the execution of experiments and data collection to bolster the validity of the approach and provide substantial support for the conclusions. Additionally, in this environment it is easier to obtain the necessary permissions and cooperation from university authorities and users to conduct experiments in an ethical and legal manner.

4.1. Real-World Case Study: University of the Basque Country (UPV/EHU)

This case study has been designed upon a previous study carried out at the Technological University of Dublin (TU Dublin) [127,128] aiming to analyze and compare contextual differences within a similar environment but across different countries.

To achieve this goal, in this case study, the data were collected at the University of the Basque Country (UPV/EHU), composed of different campuses with multiple locations. In order to have a large and varied dataset to establish the relationship between objective QoS data and subjective QoS data, faculties from different study domains were selected. In particular, the data used for this study were collected at the Bilbao Faculty of Education in Leioa, the Faculty of Economics and Business, and the Bilbao Engineering School in Bilbao during September and October 2023.

It must be underlined that, in the TU Dublin case study, Wi-Fi probes [129] were used to capture the network performance data. However, for the UPV/EHU experiment, QoS objective data were obtained directly from the access points and the Cisco Prime controller of the university Wi-Fi network.

4.1.1. Methodology Validation: Steps under Study

As mentioned in Section 3.2, the initial phase of the proposed methodology aimed to infer distinct scenarios and user profiles through unsupervised machine learning techniques, specifically clustering analysis (context extraction Figure 12). For this purpose, network-recorded data were used, which included both the user’s contextual data (such as location, scenario type, day/time/usage duration, etc.) as well as the so-called system-related influence factors (device type, operating system, etc.).

Once the distinct scenarios and user profiles were inferred, both objective and subjective information was gathered to facilitate the identification of key quality indicators (KQIs), key performance indicators (KPIs), and key performance parameters (KPPs) that may impact users´ QoE. In a more advanced stage of validation and with larger survey data, it is intended to use supervised machine learning (inductive techniques) to identify these key indicators that impact QoE (as proposed in Section 3.2). In this case study, due to the sample size, we have been able to directly induce that information.

The next step of the methodology validation addressed in this case study was the automated mapping of NP with QoE. As proposed in Section 3.2, for automating this correlation, the use of supervised machine learning algorithms (regression models, etc.) is suggested (NP/QoE correlation in Figure 12) using both objective data (KPI values) and subjective data related to user experience and satisfaction) collected through surveys (i.e., Mean Opinion Score).

Figure 12 shows the flow diagram of the whole process conducted in this case study.

4.1.2. Research Sample and Data Collection

As stated in previous subsections, both objective and subjective information from the Bilbao Faculty of Education in Leioa, the Faculty of Economics and Business and the Bilbao Engineering School in Bilbao were used to develop this case study.

Next, the method for both the objective and subjective data acquisition and the research sample characteristics is described:

• Objective data collection: Before data collection, a formal agreement between the University of the Basque Country (UPV/EHU) and the NQaS (Networking, Quality, and Security) research group was signed regarding the use of data collected from the university’s network system for research and scientific dissemination purposes. In this regard, it was committed to maintaining the security of the information and the anonymity of the participants throughout the research process. Therefore, prior to analysis, the pseudo-anonymization procedure was carried out. This process was endorsed and authorized by the university’s Data Protection Officer and Ethics Committee. As mentioned before, it is remarkable that, unlike the case study conducted at TU Dublin [127,129], at UPV/EHU, the objective data were directly obtained from the university’s network system. The data were collected through the university´s Cisco Prime system controller, which obtains data from the different access points installed throughout the buildings on the different campuses. In our case study, data were collected at the two locations mentioned in the previous section during one month.
• Subjective data collection: A survey was designed to gather subjective QoS data information (requirements, expectations, and satisfaction) and conducted with a sample size of 80 individuals. Among these participants, 58% were identified as male, while 42% were identified as female. The demographic distribution across age groups was as follows: 4% were aged above 60 years, 13% fell within the age range of 45 to 60 years, 31% were aged between 30 and 45 years, and a significant proportion of 52% belonged to the age interval of 20 to 30 years. It is pertinent to note that there were no respondents below the age of 20 years. All the participants were linked to the UPV/EHU with different roles within the institution. Specifically, 55% were students, 14% were lecturers, 15% were researchers, and 16% were support staff. This diverse representation ensured a comprehensive cross-section of perspectives from different segments of the university community.

In the next subsections, more information regarding the characteristics of the collected data is provided. It must be remarked that, as required in our university for any human-involved data experiment, the data collection process was approved at the end of July by the Ethics Committee for Research Involving Human Subjects, their Samples, and Data (CEISH) of the UPV/EHU (Approval Report M10_2023_184 granted on 24 July 2023)

4.1.3. Objective Information: Network Data Collected

In Figure 13a, a resume of the eduroam Wi-Fi network data collected during one month, from 11 September 2023, at the start of the academic year, to 8 October 2023, is illustrated.

Even though the Cisco controller deployed at the university supported Wi-Fi 6 (802.11ax), the access points along the different campuses do not support this protocol, so none of the captured data (Figure 13b) are associated with Wi-Fi 6/6e (801.11ax) on any of the channel bandwidths, which is one of the reasons that led us to extend the study to simulated scenarios. Upon witnessing this Wi-Fi experiment, one can reasonably anticipate that the current state of modern networks (i.e., Wi-Fi 6e, Wi-Fi 7, etc.) may not have reached a widespread deployment that would enable the feasibility of conducting real-world studies on these technologies.

All the data shown in Figure 13, collected from the Cisco controller, were used for the clustering process to infer the different user profiles after being pseudo-anonymized. In addition, some of the collected data (KPIs with impact on QoE of surveyed users) were also used for the supervised machine learning training of the QoS/QoE correlation process.

4.1.4. Subjective Information: Survey

As mentioned before, the data collection process, including the survey, received approval from the Ethics Committee for Research Involving Human Subjects, their Samples, and Data (CEISH) of the UPV/EHU in July 2023 (Approval Report M10_2023_184 granted on 24 July 2023). In this report, it was specified that, during the initial phase of the study, all the participants should be identified through a unique user identifier. This identifier was crucial to be able to establish the linkage between the user´s survey and the network data collected referring to the user´s specific network connection. Thus, personal/private data could not be used or analyzed, fulfilling current regulations regarding privacy and data protection. The process of pseudonymization, implemented before the data preparation and handling phases, ensured the safeguarding of personal/private information.

The survey was designed to capture personal information (gender, age, occupancy, and Internet expertise) together with the data related to the “Wi-Fi experience”, both about the user’s requirements/expectations and QoE/satisfaction with the service. Participants’ involvement consisted of two distinct phases. The initial phase consisted of a series of tests conducted using a device connected to the eduroam Wi-Fi network. These tests encompassed various services, including video calls, file downloads, performance assessments, etc. The second phase involved participants responding to an online questionnaire. Next, these phases are detailed:

• First survey phase: Participants were required to connect to the Eduroam Wi-Fi network (disabling the remaining networks) using a laptop, tablet, or any other mobile device. Then, they were required to:

  ○

  Conduct a video call using the Microsoft Teams Classic video conferencing tool installed on the device, an official platform at UPV/EHU.

  ○

  Perform a speed test and subsequently upload the obtained result to a corporate file service.

• Second survey phase: participants were required to respond to an online survey (Microsoft Forms) comprising 32 questions, categorized into three different sections: personal information (9 questions), usage patterns of the Wi-Fi eduroam network (13 questions), and experience and expectations concerning the use of the UPV/EHU Wi-Fi eduroam network (10 questions).

To ensure that the collected data contributed pertinent insights to the study, the involvement of diverse user profiles (students, researchers, lecturers, etc.) was of utmost importance. Consequently, participants were required to respond to a segment of questions relating to personal attributes (age, field of study, etc.) and concerning usage patterns, experiences, and expectations related to the Wi-Fi eduroam network experience within UPV/EHU campuses.

In the proposed methodology, subjective information is crucial for the identification of the KQIs and KPIs via supervised machine learning using the influencing factors (IF). Furthermore, this information will aid in training the system in the QoS/QoE correlation process.

4.1.5. Results Analysis

As previously mentioned, a preliminary methodology validation was conducted some time ago in another university campus scenario in the Dublin area (TU Dublin). Comparing UPV/EHU results to those of TU Dublin presented in [127], it can be concluded that network security continues to be a very important QoS criterion for eduroam users. Nevertheless, the network speed criterion has lost weight in our UPV/EHU current scenario, where only 15% consider it an indicator of great importance (with survey ratings of satisfaction level 5, using the same rating scale as the MOS), compared to 38% who believe it is just important (with survey ratings of satisfaction level 4). While the criteria of security is still considered essential for 80% of the sample. These results demonstrate the importance of the first step considered in the proposed methodology for the QoXphere model implementation: the context extraction (see in Figure 12). QoX management can be profoundly influenced by user behavior and a multitude of contextual and non-contextual factors that fall outside the control of service providers and, therefore, a preliminary analysis is crucial for identifying the influential factors associated with each specific scenario and user profile. Details of the analysis of this first step in the UPV/EHU case study can be found in next subsection.

Context Extraction: Clustering Analysis

A comparative analysis of different clustering methods is performed using K-means, DBSCAN, Gaussian Mixture Models (GMM), and Agglomerative Clustering (AGG) algorithms. Three metrics have been used to determine which of the clustering methods is the most suitable: the Silhouette Score, the Davies–Bouldin Index, and the Calinski–Harabasz Index. The results for the different scenarios are summarized in

Table 2. Cluster evaluation metrics result.

Method	Silhouette Score	Davies–Bouldin Score	Calinski–Harabasz Score
Kmeans (k = 2)	0.567	0.782	3647.610
Kmeans (k = 3)	0.454	1.018	3232.949
Kmeans (k = 4)	0.461	0.941	2646.394
eps: 0.1, min_samples: 10, clusters: 4	0.248	1.162	524.332
eps: 0.1, min_samples: 15, clusters: 4	0.056	1.265	426.260
eps: 0.2, min_samples: 10, clusters: 3	0.507	1.157	1619.141
eps: 0.2, min_samples: 15, clusters: 3	0.497	1.174	1522.969
eps: 0.3, min_samples: 10, clusters: 5	0.465	1.289	1010.286
eps: 0.3, min_samples: 15, clusters: 3	0.530	1.084	1693.434
eps: 0.4, min_samples: 10, clusters: 3	0.539	1.111	1563.499
eps: 0.4, min_samples: 15, clusters: 3	0.538	1.057	1598.613
GMM (clusters: 2)	0.391	1.388	1747.865
AVG (clusters: 2)	0.461	1.710	2646.394

. Although only the results of the evaluation metrics applied to the Bilbao Engineering School scenario are shown, they are scalable to other scenarios. The results of analysis of the evaluation metrics lead to the conclusion that K-means clustering model provides the highest accuracy in discerning distinct scenarios and user profiles in this case study.

Based on the various influence factors (context, human, and system IFs) captured through the collected objective and subjective data, the KQIs and corresponding KPIs relevant to our case study were identified. It is worth noting that this step is also proposed for implementation with ML techniques in real scenarios or future validation case studies. Once the KPIs to be considered are determined (packets sent/received in this case study), the elbow method is applied to calculate the number of clusters. As a result, the decision was made to apply the K-means clustering algorithm with two clusters (Figure 14), which is supported by the evaluation metrics (Table 2).

The results of the application of the K-means method in the three scenarios (Engineering School, Education Faculty, and Faculty of Economics and Business) are shown in Figure 14. There, cluster 1 gathers those instances with low packet reception and transmission and cluster 2 gathers those with higher values. In all the scenarios studied, these two groups are present; however, upon closer examination of the figure, distinct variations between them become apparent. Within cluster 1, the user profile is likely linked to passive users who exhibit limited activity on the network, primarily relying on mobile devices. Conversely, cluster 2 represents users actively engaging with the network, primarily utilizing laptops. Notably, the figure shows a significantly higher number of users in the Engineering School, which aligns with expectations.

According to UPV/EHU data, roughly 8000 students pursue engineering studies, while only around 3200 opt for humanities studies, and approximately 15,000 students are enrolled in social and legal science, a correlation that is evident in our study. These statistics correspond to the number of degree programs and staff available in the respective academic centers. In examining the distribution of user types within various areas of each building, a cross-referencing approach by correlating the identified clusters with the geographical placement of utilized access points has been carried out. This analysis reveals that, in the case of cluster 2, the majority of devices are situated within housing study facilities regions (libraries or open study areas). In addition, a clear variation in the boundaries between the two clusters across the three scenarios is shown in Figure 14, indicating different patterns of behavior in each case. The evaluation metrics for each scenario are shown in

Table 3. K-means (k = 2) evaluation metrics.

Location	Silhouette Score	Davies–Bouldin Score	Calinski–Harabasz Score
Bilbao Engineering School	0.567	0.782	3467.610
Bilbao Faculty of Education	0.565	0.770	2130.870
Faculty of Economics and Business	0.575	0.800	1434.958

; these values indicate that the clusters are adequate and consistent for the three scenarios.

When compared with the previous study at TU Dublin, it becomes apparent that the user profiles were not significantly different, as in Dublin, two clusters were identified with either high or low transmission rates, which aligns with the results at the UPV/EHU.

QoS/QoE Correlation: Supervised Machine Learning

In order to develop a precise and efficient QoS/QoE correlation model for the QoE estimation (eliminating the need for surveys), the relationship between the objective QoS, in terms of KPIs/KPPs, and the QoE, in terms of MOS (Mean Opinion Score) must be established (NP/QoE correlation model in Figure 12). A comprehensive correlation analysis was conducted to achieve this, comparing different supervised machine learning models and encompassing all parameters within the dataset.

KQIs and corresponding KPIs of influence in our case study were identified from the subjective data collected (again, note that this step is also proposed to be implemented with ML techniques in real scenarios or future validation case studies). The heat map of the correlations between pairs of KPIs (objective parameters) is shown in Figure 15, where the subjective value of the MOS, obtained from the surveys, has also been included. The correlation matrix refers to the entire sample, but it is representative of the correlation matrices obtained for each of the scenarios independently. The correlation values between the Average Throughput and MOS variables for each of the centers are shown in

Table 4. Throughput and MOS correlation coefficients.

Location	Correlation Coefficient
Bilbao Engineering School	0.72
Bilbao Faculty of Education	0.66
Faculty of Economics and Business	0.57

. Throughput was selected as the KPI of major influence in our case study. Throughput is a key performance indicator that is widely used in network management standards to measure network capacity. Ongoing standards efforts aiming to improve throughput definition emphasize its essential role in measuring network capability. Specifically, in this case study the parameter used was the average value of the session throughput that provides an average measure of the connection performance during the time that the user has been connected to the Eduroam Wi-Fi network. As in most QoE studies, the MOS value was designed to evaluate the QoE. Once NP/QoE indicators of influence were designed, the NP/QoE correlation procedure was developed:

• Data Preparation: The survey data collected were organized, readied and aligned with the associated objective data extracted from the network controller. Additionally, user and device identifiers were securely established, ensuring the preservation of anonymity and adherence to privacy regulations. The dataset was further prepared by removing outliers and the application of normalization techniques. As depicted in Table 4, the computed correlation coefficient reveals a moderately positive relationship in both scenarios. This implies that, as expected, as throughput escalates, the quality of experience (QoE) similarly sees an improvement.
• Next, the collected data were partitioned into distinct subsets, with one allocated for training and the other for validation. The performance of the model on both the training subset and the validation subset was evaluated using the Mean Squared Error (MSE) and the Root Mean Squared Error (RMSE) metrics. These metrics are useful to detect underfitting or overfitting issues, since both conditions result in low performance model and training errors. In this preliminary pilot study, conducted with a limited dataset of just eighty surveys, a division of 20% for training and 80% for model validation was employed. Following this, an input dataset for model validation was randomly selected.
• Correlation Analysis: A comprehensive correlation analysis was undertaken, encompassing the research variables, which include QoS parameters and data related to QoE collected through questionnaire responses. A statistical analysis was also performed on the research sample, encompassing the calculation of basic statistical indicators linked to respondents’ answers. In addition, various metrics and techniques to evaluate the ML models were employed, enabling assessment of their performance and accuracy. The main metrics chosen were Mean Squared Error (MSE), Mean Absolute Error (MAE), and the coefficient of determination (R²). The modeling process was executed using both linear regression and feedforward neural networks. The Polynomial Regression method was also tested, obtaining evaluation metrics within the following ranges: 0.27 < MSE < 0.40, 0.4 < MAE < 0.5, and 0.4 < R² < 0.8. However, the model predicted values that were out of range of MOS, so different adjustments were made by modifying the degrees and applying regularization—all of them without achieving a trend curve that complies adequately. The following subsections show the results and evaluation of linear regression and feedforward neural network models.

  ○

  Linear regression: This method was developed to unveil the equation of the line that optimally characterizes the data’s relationship, with the objective of minimizing the sum of squared disparities between the real values and those estimated by the regression line. In

  Table 5. Evaluation metrics for Linear Regression.

  Location	MSE	MAE	R2
  Bilbao Engineering School	0.198	0.349	0.493
  Bilbao Faculty of Education	0.346	0.458	0.637
  Faculty of Economics and Business	0.30	0.42	0.80

  , the comprehensive evaluation metrics for this model can be observed.

As shown in Table 5, the Mean Squared Error (MSE) value was relatively low, signifying that the model could be considered acceptable. However, when considering this value alongside the other metrics, it suggested that there was potential for further optimization of the model. The best evaluation metrics and, consequently, the best model performance were obtained in the Faculty of Economics and Business, with low MSE and MAE and the highest R² score. On the other hand, the Bilbao Engineering School model showed an intermediate performance, and the poorest fit of the model. Overall, these results suggest that the model displays different behavior and performance at each location.

In addition, the evaluation metrics for the training and test subsets in the different scenarios are shown in

Table 6. Evaluation metrics for Linear Regression (training-fitting).

Location	Training			Fitting
	MSE	MAE	R2	MSE	MAE	R2
Bilbao Engineering School	0.31	0.45	0.71	0.28	0.48	0.69
Bilbao Faculty of Education	0.69	0.66	0.66	0.66	0.79	0.59
Faculty of Economics and Business	0.32	0.44	0.82	0.23	0.34	0.64

. The evaluation values for both subsets are quite close, and the results are reasonably good, so we can deduce that the model is not overfitted and achieves good prediction results.

However, the results of the K-Fold Cross-Validation, shown in

Table 7. K-Fold Cross-Validation results.

Location	R² Scores for Each Fold	Mean R² Score	Standard Deviation of R² Scores:
Bilbao Engineering School	[0.377, 0.01, −2.00, −1.51, 0.40]	−0.544	1.014
Bilbao Faculty of Education	[−6.31, −5.5, −8.97, 0.43, −44.37]	−12.96	16.005
Faculty of Economics and Business	[−0.42, 0.19, −0.25, 0.30, −0.49]	−1.732	3.391

, indicate a low performance of the model. This may be due to different reasons, including the possibility that the model may not be adequate or that the relationship between the variables is not linear.

The MOS estimation lines and real data for the three scenarios are shown in Figure 16. The Bilbao Engineering School trend line has a lower slope, which suggests that users in this faculty are more demanding than in others. This is evident in Figure 16, where QoE improves more rapidly, and users at the Education Faculty report higher Mean Opinion Score (MOS) values for lower throughput than users at the Engineering School or the Faculty of Economics and Business.

○

Feedforward Neural Network: This represents a more sophisticated modeling approach, well-suited for handling the intricacies of the case study’s dataset, which may entail non-linear relationships. Considering the relatively small dataset available, it was decided to use a straightforward sequential neural network structure comprising three layers: 64, 32, and 1 neuron, respectively. The Adam optimizer was used to compile the model, which dynamically adapts the learning rate during training, and the MSE loss function, which calculates the average of squared differences between the labels and predictions.

In addition, different neural network architectures and different hyperparameters have been evaluated (different levels, splits, etc.), trying to minimize the loss function. For model evaluation, the metrics MSE, MAE, and R² were calculated. In general, the evaluation results of the different RNN models have been positive, with low MSE and R² scores above 0.5, indicating an acceptable model fit where the predicted values are close to the real ones. The results presented in

Table 8. Evaluation metrics for Feedforward Neural Network.

Location	MSE	MAE	R2
Bilbao Engineering School	0.157	0.403	0.752
Bilbao Faculty of Education	0.201	0.364	0.875
Faculty of Economics and Business	0.315	0.486	0.647
All Dataset	0.133	0.312	0.916

refer to the model that has offered the best evaluation results, which outperforms the linear regression approach. The model achieves a coefficient of determination (R²) close to 1 when analyzing the dataset as a whole, while the MSE is very close to 0 and the MAE is 0.312. These results indicate that the model is not only making accurate predictions (low MSE and MAE), but also explains most of the variability in the MOS scores. This validates that the prediction model is performing very well, fits the data excellently, and is highly reliable for future predictions.

Furthermore, reviewing the learning curves of the model during training helps to identify underfitting or overfitting issues or to validate the dataset as representative. The loss curves for the Faculty of Economics and Business, to evaluate the prediction model, are shown in Figure 17 and are representative of all the scenarios. The validation loss curve decreases to a stability point, and there is a small gap between the validation and training loss curves, suggesting that the learning curves are well-fitted.

Figure 18 illustrates the correlation between throughput and the predicted MOS (QoE) in the three faculties and another trend line representing the curve for the MOS (QoE) estimation of the entire sample dataset. It is worth noting the strong positive correlation between the two variables. This graph demonstrates the neural network’s capacity to capture logarithmic patterns. Furthermore, it becomes evident that the MOS value remains stable beyond a certain throughput threshold. Consistent with the findings from the linear regression model, it is confirmed that the estimated maximum MOS value is reached earlier in the Faculty of Education, indicating that users in this faculty are less demanding.

The analysis of these curves allows us to extract more information about the users. Moreover, the trend line for the Bilbao Engineering School, which shows the slowest growth, combined with the users’ academic profile, suggests that the users are more critical and that they have higher expectations regarding the Wi-Fi service. The trend line for the Faculty of Economics and Business is somewhat different, as participants in the sample from this faculty have reported more negative evaluations. This is probably due to previous experiences with the service (students have frequently complained about the Wi-Fi service in this faculty), which also influences their perception.

4.2. Simulated Case Study: Modern Networks Context Extraction

As mentioned in the previous subsection, the deployment of modern networks on a wide scale is not extensive enough to justify conducting real-world studies of these technologies. For this reason, it was decided to initiate a new case study characterizing modern networks in a simulated environment with NS3.

Before simulating 5G or 6G scenarios, initial experiments were carried out using IEEE 802.11ax (Wi-Fi 6) technology. In this study, several simulation patterns have been developed and key performance indicators have been inferred. In addition, an analysis of the aptitude of clustering techniques has been carried out, focusing on validating the first step of the proposed methodology: context extraction to identify different scenarios and user profiles.

The development of the simulated case study for the methodology validation is still in its early phases but is expected to be of great help in validating the proposed methodology.

5. Conclusions

The advent of modern networks like 5G and 6G, together with the emergence of transformative applications, underscores the need for comprehensive global quality of service (QoX) management models that extend beyond technical aspects. User behavior shifts and evolving application requirements emphasize the growing importance of holistic quality management that encompasses not only network performance and QoS but also quality of experience (QoE) and quality of business (QoBiz).

In response to these challenges, this article introduces a machine learning-based methodology for the deployment of a QoS management model named QoXphere. The proposed methodology aligns with international standards and is built upon the ITU-T’s Rec. Y.3170 requirements of machine learning-based QoS assurance for modern networks. Therefore, the approach offers a promising method to dynamically manage resources in a way that enhances user satisfaction while optimizing provider benefits.

Although the validation of this methodology is still in the preliminary stages, initial case studies conducted on university campuses using Wi-Fi networks hint at the potential efficacy of the approach in flexibly managing QoX. Simulations with NS3 are being designed to validate the methodology in the dynamic and diverse landscape of modern networks.