Serendipitous, Open Big Data Management and Analytics: The SeDaSOMA Framework ^†

Abstract

This paper presents and delves into the architecture and intricacies of SeDaSOMA, a sophisticated framework supporting Serendipitous, Data-as-a-Service-oriented, Open big data Management and Analytics. SeDaSOMA meticulously addresses the multifaceted challenges inherent in open big data management and analytics. SeDaSOMA stands as a testament to the evolving landscape of big data management and analytics, embodying a commitment to harnessing advanced functionalities through a synthesis of innovative research findings and cutting-edge tools. In the context of this framework, the paper not only elucidates its structural components but also underscores its pivotal role in facilitating the seamless integration, processing, and analysis of massive and diverse datasets. By providing a comprehensive overview of SeDaSOMA, this paper contributes to the ongoing discourse within the field of big data management and analytics, shedding light on the intricate interplay between technological innovation and practical application. Moreover, as a complement to the discussion on SeDaSOMA, the paper offers a critical exploration of the emerging topics in the realm of big data research. By delineating current state-of-the-art methodologies and forecasting future research trajectories, this overview enriches the scholarly dialogue surrounding the evolving landscape of big data management and analytics, offering insights into the broader implications and potential advancements in the field.

1. Introduction

The amalgamation of big data management and analytics represents a significant and evolving area of interdisciplinary research, which draws upon diverse studies spanning database systems, data warehousing, data mining, and machine learning (e.g., [1,2]). This integrated approach synthesizes various research endeavors conducted over time. Notably, the application of big data management and analytics has witnessed a proliferation across a spectrum of domains, ranging from smart cities to social networks, from sensor networks to intelligent transportation systems, and from supply-chain analysis to economic development protocols. For instance, in smart cities, big data management and analytics facilitate efficient resource allocation and urban planning [3]. Similarly, in social networks, these methodologies enable insightful analyses of user behavior and network dynamics [4]. Moreover, their application extends to domains such as sensor networks [5] where they contribute to data-driven decision-making processes and to intelligent transportation systems [6] where they enhance traffic management and optimize route planning. Additionally, in areas like supply-chain analysis [7] and economic development protocols [8], big data management and analytics play pivotal roles in optimizing operational efficiency and fostering sustainable growth strategies. This multidisciplinary convergence underscores the pervasive impact and versatile applications of big data management and analytics across diverse domains.

Indeed, early research, exemplified by OLAP data cube processing [9,10] for big data management and OLAP mining [11,12,13,14] for big data analytics, has elucidated that a direct transference of findings to the context of big data is not feasible due to emergent challenges. Consequently, the necessity for innovative techniques and algorithms arises [15,16]. This imperative is further underscored by the escalating requirement for intelligent big data applications within the information and communication community [5,17,18,19].

Following this trending research topic, this paper presents the architecture of a reference framework, namely SeDaSOMA, which supports Serendipitous, Data-as-a-Service-oriented, Open big data Management and Analytics. The main goal of this framework is to provide advanced big data management and analytics on the basis of innovative research achievements and next-generation big data tools.

SeDaSOMA presents a sophisticated software architecture tailored for serendipitous, Data-as-a-Service (DaaS)-oriented management of big data and predictive analytics purposes. The architecture integrates a comprehensive suite of models, techniques, and algorithms designed to acquire, represent, manage, and secure big data within modern Cloud-based DaaS paradigms (e.g., [20]). Specifically, it aims to establish a novel data marketplace environment and foster an open big data ecosystem capable of supporting scalable big data analytics. To achieve this goal, the framework leverages innovative methods for big data query answering such as the following: (i) Distributed query processing; (ii) Approximate query processing; (iii) Query optimization for heterogeneous data sources. In addition, it leverages context-aware approaches such as the following: (i) Semantic data integration; (ii) Adaptive analytics frameworks; (iii) Context-sensitive anomaly detection). Furthermore, it leverages big data preparation techniques tailored for developing effective big data analytics solutions such as the following: (i) Data cleansing and preprocessing; (ii) Feature engineering and selection; (iii) Scalable data transformation pipelines. The SeDaSOMA reference architecture contains several main layers that represent the functional and procedural basis of the framework, which is similar to other initiatives in the field (e.g., [21]). Typically, these functions and procedures are destined for data-intensive tasks and also for a wide range of Cloud-aware big data vertical applications that vary in a range determined by the main areas of challenges in the current information community, which are specified by the European data management guidelines by the European Commission [22], with particular reference to the following: (i) Social big data management for workplace safety and health; (ii) Integration of (big) bank data and (big) customer data for supporting big data intelligence; (iii) Big data advertisement on the Web for opportunity finding. The latter only represents some case studies that have been considered as critical examples in modern big data settings.

This paper is organized as follows. Section 2 describes some related works relevant to our research. In addition to this, we provide an analysis that focuses on the main contributions of the SeDaSOMA framework over conventional state-of-the-art approaches. In Section 3, we provide a detailed description and composition of the SeDaSOMA framework. In Section 4, we focus on providing a critical overview of some emerging topics of big data research, which are close to our work, by highlighting current state-of-the-art and future research efforts in the field. In Section 5, we introduce a practical implementation of SeDaSOMA that focuses on big clinical data. Finally, in Section 6, we give the conclusions of our research work along with the future works that we plan to undertake to extend this work. This paper relevantly extends the paper [23] where we first introduced the framework idea and the main design guidelines. Other past contributions that are now deeply extended in this work focus on privacy-preserving and approximate big data management [24] and integration of multidimensional modelling and Artificial Intelligence (AI) [25].

2. Related Work

In recent years, the management and analysis of open big data have garnered significant attention in both academia and industry (e.g., [26]). In this Section, we provide a comprehensive review of relevant literature by focusing on key developments and methodologies in the field of Open Big Data Management and Analytics (e.g., [27]).

In [28], authors adopt a complexity-theory approach and suggest that organizations develop different approaches to leverage their big data analytics resources toward the attainment of organizational goals [29]. They build on a sample of 175 survey responses from IT managers in Greek firms and examine the patterns of big data analytics resources that lead to high levels of performance. Authors also apply a configurational approach through a novel methodological tool named fsQCA, which allows the examination of such complex phenomena and the reduction of solutions to a core set of elements. In addition, the authors examine three case studies to uncover how these elements, as well as other core enablers or inhibitors, can emerge and how they coalesce and impact performance.

Ref. [30] provides a valuable systematic literature review on Big Data across business management between 2009 and 2014. Although the field is in its earliest stages of scholarly development, authors found clear evidence of the energy and increasing interest focused on Big Data studies in business. Numerous studies have examined Big Data initiatives in organizations. These include the Big Data revolution in corporate strategies and management. The clear purpose is to understand and offer a conceptual framework as well as define an empirical interpretation of a Big Data methodological approach in organizational Competitive Intelligence (CI) cycles. To this end, the authors provide a comprehensive description of the evidence received from the selected organizations on Big Data practices as well as analytical approaches in corporate CI processes. In fact, they introduce basic Big Data like analytics, competitor analysis, SWOT, segmentation analysis, and five forces analysis; however, we have not used this in real time on larger data. The authors also employ advanced text mining, natural language processing, etc., which are all related to Big Data. Overall, the research findings provide a clear understanding that firms have yet to consider Big Data technologies in CI processes; these results have contributed to starting the building of the process for Big Data in organizational CI cycles.

Ref. [31] recognizes that the emergence of cloud computing techniques and big data processing models, such as MapReduce, has made big data analytics and knowledge management a trending topic in the research world. However, regardless of the wide adoption of applications built following the MapReduce paradigm on public Clouds, the lack of trust in the participating virtual machines deployed on these Clouds resulted in a blockage in further adopting these applications. Therefore, the authors extended the existing hybrid Cloud MapReduce architecture to Multiple Public Clouds in their paper. As a result, they have proposed IntegrityMR, i.e., an integrity assurance framework for big data analytics and management applications based on this type of architecture. Furthermore, the authors investigated the integrity check techniques obtained at two alternative software layers: the MapReduce task layer and the application layer. In addition, based on Apache Hadoop MapReduce and Pig Latin, they designed and implemented their system at both layers and evaluated it by performing a series of experiments using common big data analytics and management tools such as Apache Mahout and Pig on commercial, public Clouds (Amazon EC2 and Microsoft Azure) and local cluster environments. The results obtained from these experiments following the task layer approach show high integrity (98% with a credit threshold of 5) with non-negligible performance overhead (18% to 82% extra running time compared to the original MapReduce).

In [32], the authors describe the Ophidia project, i.e., a research work that deals with big data analytics requirements, problems, and challenges for e-science. Where the authors present Ophidia, an analytics framework responsible for atomically processing, transforming, and manipulating array-based data, also capable of running large clusters of analytics tasks performed on big datasets. Additionally, the paper describes the design principles, algorithm, and most important aspects in the implementation of the Ophidia analytics framework, along with presenting some experimental results related to some data analytics operators in a real cluster environment.

Ref. [33] argues that Cloud computing and big data analysis are gaining lots of interest across a range of applications, including disaster management. These two technologies together provide the capability of real-time data analysis not only to detect emergencies in disaster areas but also to rescue the affected people. The paper presents a framework that supports emergency event detection and alert generation by analyzing the data stream, which includes efficient data collection, data aggregation, and alert dissemination. One of the goals of such a framework is to support an end-to-end security architecture to protect the data stream from unauthorized manipulation as well as leakage of sensitive information. The proposed system provides support for both data security punctuation and query security punctuation. The paper presents the proposed architecture with a specific focus on data stream security. It also briefly describes the implementation of security aspects of the architecture.

In [34], the authors focus their attention on decision making in the context of natural disaster management, where several challenges that need to be faced exist. When disasters happen, the government, being a response organization has to take immediate and accurate decisions in order to rapidly assist and effectively recover for the victims involved. This paper aims to adopt strategic decision making in government regarding disaster management by employing Big Data Analytics (BDA) techniques. BDA methodologies are integrated as solutions for managing, employing, maximizing, and displaying climate change data insights when dealing with water-related natural disasters. NAHRIM is a government agency responsible for elaborating research on water and its environment. The latter proposed a natural disaster management BDA framework for utilizing NAHRIM historical and simulated projected hydroclimate datasets. The development of this framework aims at assisting the government in the decision-making process regarding natural disaster management by fully employing NAHRIM datasets. This BDA framework consists of three stages; Data Acquisition, Data Computation, and Data Interpretation and seven layers; Data Source, Data Management, Analysis, Data Visualization, Disaster Management, and Decision. And it is anticipated that this framework will have an effective impact on the prevention, minimization, preparation, adaptation, response, and recovery of water-related natural disasters.

Ref. [35] considers that advanced digitalization, together with the rise of disruptive Internet technologies, are key enablers of a fundamental paradigm shift observed in industrial production. This is known as the fourth industrial revolution (Industry 4.0), which proposes the integration of the new generation of ICT solutions for the monitoring, adaptation, simulation, and optimization of factories. With the democratization of sensors and actuators, factories and machine tools can now be sensorized, and the data generated by these devices can be exploited, for instance, to optimize the utilization of the machines as well as their operation and maintenance. However, analyzing the vast amount of generated data is resource-demanding both in terms of computing power and network bandwidth, thus requiring highly scalable solutions. The paper presents a novel big data approach and analytics framework for the management and analysis of machine-generated data in the Cloud. It brings together standard open-source technologies and the exploitation of elastic computing, which, as a whole, can be adapted to and deployed on different Cloud computing platforms. This enables the reduction of infrastructure costs, the minimization of deployment difficulty, and the provision of on-demand access to a virtually infinite set of computing power, storage, and network resources.

Finally, in [36], the authors recognize that due to the increased usage of ICT technologies in the field of smart cities, the data generated, as a result, has increased manifold. This data is heterogeneous in nature, as it varies with respect to time and exhibits the properties of all the essential V’s for big data. Therefore, to handle such an enormous amount of data, various big data processing techniques are required. To cope with these issues, the paper presents a tensor-based big data management technique to reduce the dimensionality of data gathered from the Internet-of-Energy (IoE) environment in a smart city. The core data is extracted from the gathered data by using tensor operations such as matricization, vectorization, and tensorization with the help of higher-order singular value decomposition. This core data is then stored on the Cloud in the reduced form. After reducing the dimensionality of data, it is used to provide many services in smart cities, and its application to provide Demand Response (DR) services has been discussed in the paper. For this purpose, Support Vector Machine (SVM)-based classifier is used to classify the end-users (residential and commercial) into normal, overloaded, and underloaded categories from the core data. Once such users are identified as taking part in the DR mechanism, utilities then generate commands to handle their DR in order to alter load requirements so that the overall load is optimized. Results obtained on the Open Energy Information and PJM dataset clearly indicate the supremacy of the proposed tensor-based scheme over the traditional scheme for DR management.

Innovations of SeDaSOMA over Conventional State-of-the-Art Approaches

The SeDaSOMA framework introduces some important innovations over conventional state-of-the-art approaches, like those provided and discussed in our literature analysis. These innovations are fully discussed in Section 3, along with all the specific properties of SeDaSOMA, but here, we highlight those that fill the gap with respect to literature contributions. In the following, we summarize these innovations:

• Support for Big Data Crowdsourcing: One of the most prominent characteristics provided by SeDaSOMA is represented by the fact that its big data layer is alimented via crowdsourcing methodologies, like in some recent initiatives (e.g., [37]—contrary to this, in conventional approaches, big data repositories are alimented via methods that, essentially, are inspired by traditional ETL (Extraction, Transformation and Loading) procedures (e.g., [38]); obviously, the usage of crowdsourcing methodologies ensures more dynamicity in the data sources, more scalability, and more heterogeneity;
• Support for Big Data Marketplacing: SeDaSOMA provides support for big data marketplacing (e.g., [39]), meaning that, after that big data within the SeDaSOMA data layer are consolidated, they are not exposed to the big data analytics layer via conventional methods (e.g., publish-subscribe, push-down, ETL, etc.) but, rather, via the innovative data marketplace model, according to which client users/applications can access a marketplace of (consolidated) big data sources and select the ones that are more appropriate and “convenient” to them based on their big data analytics goals, preferences, workload issues, and so forth;
• Support for Context-Aware Big Data Processing: In addition to the above-listed innovations, the SeDaSOMA framework makes use of another really relevant feature, i.e., the approach of processing big data in dependence on the context which respect to which they are produced, located, elaborated, and consumed. This is a topic that is raising a great deal of attention now (e.g., [40]), as it overall allows us to achieve better intelligent big data processing, which turns to play a critical role with respect to specific and advanced challenges of big data, such as big data understanding and big data fruition;
• Support Big Data Preparation for Analytics: Finally, SeDaSOMA introduces the nice amenity of applying data preparation techniques (e.g., [41]) for effectively and efficiently supporting big data analytics tasks. Indeed, due to the well-known 3V properties of big data, big data cannot be processed by analytical tasks as they are, but they indeed need specific data preparation solutions (e.g., normalization, scaling, and so forth).

3. SeDaSOMA’s Anatomy

In Figure 1, we present the layered architecture of the framework SeDaSOMA. The main challenge this framework faces is to provide and experiment with a novel and innovative vision for big data management and analytics that surpasses traditional approaches, address issues regarding scalability and responsiveness of analytics over big data, and achieve novel paradigms, mechanisms, and schemes in the context of critical Cloud-aware big data vertical applications. In turn, the latter are associated with different major areas of current information community challenges specified by the European guidelines.

The architecture of SeDaSOMA is typically nested and multi-layered, where each layer consists of multiple components, as shown in Figure 1. SeDaSOMA includes the following four layers:

• Big Data Source Layer, which locates and manages different sources of big data, which are heterogeneous in nature, distributed, and streamed inherently.
• Big Data Repository and Provisioning Layer, which stores and processes big data, then transmits it to the higher layers.
• Big Data Analytics Layer, which extracts useful knowledge from big data by running complex and scalable analytics over these huge amounts of data.
• Big Data Application Layer, which includes Cloud-aware big data vertical applications that completely rely on the underlying layers. These applications focus on the main domains of modern information community challenges specified by European Union data management guidelines.

Each layer is associated with a particular set of big data management and analytics challenges (that, however, has already been investigated in actual literature—e.g., [42]); all aim to attain the same particular goal, which is to achieve innovation and advancements in critical knowledge and research, along with authoritative implementations by employing ad-hoc Cloud-aware vertical applications over big data. Additionally, the SeDaSOMA framework highlights the organization and relationships between the different layers within its composite architecture, which is considered innovative by itself, as it introduces a novel and sustainable vision of big data management and analytics that has no similarities in the current literature.

3.1. SeDaSOMA Components

In this part, we focus on the layers of the SeDaSOMA framework, where each layer consists of several components, as follows.

Big Data Source Layer, which contains the following components:

• Physical Resource Management, which handles the physical aspects of resources that are capable of generating data streams (e.g., sensors and datacenters) by diving into and exploring innovative aspects such as noise-detection and cleaning, stream pipelining, resource/stream synchronization, etc. This can be further extended by checking out other advanced topics such as designing high-performance infrastructures that solve the traditional issues regarding big data streams, including the common 3Vs of big data, i.e., Volume, Variety, Velocity.
• Heterogeneous Distributed Big Data Stream Management, which is responsible for managing big data (stream) at the framework’s input interface, where big data is considered as data objects on top of the underlying physical resources (e.g., sensors and datacenters that are capable of generating big data streams). The goal here is to overcome the following major challenges: (i) having to deal with big-data’s streaming nature; (ii) having to deal with multi-rate arrivals and inter-arrivals; (iii) having to deal with the massive amounts of (streaming) big data, etc.
• Big Data Crowdsourcing, which tackles the problem of big data collection using the innovative crowdsourcing paradigm. This aims to develop models, techniques and algorithms for applying this novel paradigm to big data collection/acquisition problems.

Big Data Repository and Provisioning Layer, which contains the following components:

• Big Data Storage, which is a major component of the SeDaSOMA framework. It tackles big data representation and storage problems in particular as scalability presents one of the key requirements. In this context, several challenges exist, which range from advanced data structures for big data storage to big data indexing, from big data partitioning techniques to innovative data management policies and regulations focusing on elastic big data storage solutions, etc.
• Big Data Warehousing, which ensures the reliability of data management within the SeDaSOMA framework, is a key feature of warehousing methodologies and solutions due to the complex nature of big data. In our case, these methodologies are captured and implemented by the Big Data Warehousing component in combination with the innovative Cloud-based paradigm, which is built on top of the Big Data Storage component and modern NoSQL architectures (e.g., in distributed settings) that are used in the implementation of this component. This component addresses the following main issues: (i) multidimensional big data models, which are critical for supporting big data analytics; (ii) compressed big multidimensional data representations; (iii) MapReduce-based big data warehousing, etc.
• Secure Big Data Processing, which represents a major concern of the SeDaSOMA framework as it is critical to guarantee the security of big data stored in the Big Data Warehousing component; this is because these (big) data are accessed and processed within an open environment, i.e., the Big Data Marketplace component. Therefore, as a result, the SeDaSOMA framework is required to incorporate ad-hoc solutions in order to secure big data processing. The Secure Big Data Processing component directly interfaces with the Big Data Warehousing component and achieves this goal. In this context, some of the relevant existing challenges are as follows: (i) ensuring scalability and security when accessing big data; (ii) implementing innovative big data encryption methodologies; (iii) defining novel techniques for big data provenance, etc.
• Big Data Marketplace, the SeDaSOMA framework contains innovative characteristics because its core data layer provides big data to higher layers and consumer (Cloud-aware) applications and makes it available by employing a novel data marketplace (DaaS) paradigm instead of using a traditional big data fruition scheme. This paradigm handles modern Cloud computing environments and anticipates an applicative setting that uses service-oriented primitives in order to make big data available for consumer applications. The main goal of this component consists of making the big data stored in the Big Data Warehousing component available to consumer applications following the DaaS paradigm, all while ensuring security (which is guaranteed by the Secure Big Data Processing component). Several open problems emerge in this context: from innovative service-oriented big data provisioning to designing models for the big data marketplace, from challenges related to scalability to Cloud-compliant heterogeneity problems, etc.

The Big Data Analytics Layer consists of the following components:

• Big Data Query Answering: this component addresses challenges related to implementing effective and efficient algorithms that support query answering over big data, along with scalability issues. The traditional query answering algorithms that have been proposed and developed in the context of traditional data-intensive scenarios (e.g., very large databases) are not capable of processing large amounts of big data; therefore, as a result, new techniques need to be introduced. The major relevant challenges regarding this concept are the following: (i) the use of data compression paradigms to enhance query answering performance while ensuring the accuracy of answers; (ii) evaluating preference-based query processing techniques and how they handle big data; (iii) scalability problems related to querying big data, etc.
• Context-Aware Big Data Processing: As highlighted before, the SeDaSOMA framework aims primarily at providing powerful data-intensive analytics in next-generation big data applications, where these analytics can be exploited based on the fortunate Cloud computing paradigm. Following this concept, it is critical to incorporate context-aware methods in order to provide “the right (big) data to the right application” due to the large amounts and the strong heterogeneity of big data. This issue has become very relevant lately, especially due to the strong relationship among topics such as advertisements and recommendations in big data. In the case of our SeDaSOMA framework, we guarantee this requirement via the Context-Aware Big Data Processing component. The major relevant challenges regarding this context are as follows: (i) context-awareness in the proposed algorithms and techniques for big data; (ii) preference-based big data processing; (iii) scalability in context-aware big data processing, etc.
• Big Data Preparation for Analytics, which is a major challenge within the process of designing and executing analytics over big data, i.e., how to perform pre-processing over big data in order to make more effective and more efficient analytics. In fact, it is a critical issue due to the characteristics of big data, such as the extremely massive size and strong heterogeneity. In addition, the analytics become irrelevant (e.g., in terms of “responsiveness”) due to the lack of a “unifying” schema. The Big Data Preparation for Analytics component is responsible for providing solutions that fulfil some technical needs in this context, such as refining existing data, reducing unnecessary data, normalizing data, and successfully extracting attributes and features from big data that guide the analytics process. Moreover, since these requirements can only be answered during the analytics phase, the Big Data Preparation for Analytics component needs to provide on-demand preparation primitives in order to be able to respond to requests from the analytics component. Finally, it is mandatory that all preparation primitives should be used in a privacy-preserving manner, especially when managing user data. SOLID is an ecosystem that we plan to refer to in order to handle and overcome the challenges of privacy (e.g., [43,44]).
• Scalable Big Data Analytics, which is the component that ensures the main goal of the whole SeDaSOMA proposal, i.e., supporting scalable big data analytics. To this end, the Scalable Big Data Analytics component addresses the problem of defining novel paradigms for next-generation analytics that are characterized by high responsiveness and high scalability. This component plays a critical role as Cloud-aware big data vertical applications rely on the SeDaSOMA framework (being devoted to assessing and possibly showing the effectiveness and the reliability of the framework), and on successfully exploiting the results and the interactions of analytics in order to attain their respective applicative goals. Several relevant research problems can be faced regarding this aspect, such as the following: (i) data-intensive analytics, i.e., analytics interacting with large-scale repositories such as big data; (ii) declarative against procedural analytics; (iii) quality-aware analytics and measures of the “quality” of analytics; (iv) responsiveness issues related to analytics; (v) scalability issues related to analytics, etc.

The Big Data Analytics Layer plays the role of the principal underlying layer in several Cloud-aware big data vertical applications where the attention is focused on how to assess the effectiveness and reliability of the SeDaSOMA framework—to which such applications are exposed. These applications are strongly dependent on the Cloud computing paradigm, and in particular, they make use of the underlying analytics presented by the SeDaSOMA framework with the goal of attaining their respective applicative results, even if depending on user/environment requirements.

With respect to the SeDaSOMA framework, there exist several major fields that critical applications of actual information society challenges specified by the European data management guidelines can fall into, such as the following:

• Social Big Data Management for Workplace Safety and Health, where, regarding vertical application, it focuses on the “serendipitily” issue when exploring and integrating (big) data collected from different media sources such as blogs, social networks, emails, forums, communities, etc., while integrating and mining such data in order to share this data among users, along with successfully exploiting it for workplace safety and health. In this case, it will be necessary that people play an “active” role, meaning that they are able to share their opinions, criticisms, and positive/negative opinions and, therefore, create a “real” (big-data-based) community with powerful analytics capabilities.
• Integration of (Big) Bank Data and (Big) Customer Data for Supporting Big Data Intelligence. Vertical applications in this context aim to improve the “on-line” experience of their customers through typical banking services delivered via different types of devices. Both types of such data (i.e., bank data and customer data) are naturally large and also introduce the common characteristics of big data (i.e., 3Vs of big data). These applications will also be exposed to dealing with heterogeneous types of data, such as location data related to customers, and services, such as health services that may take advantage of their integration with bank data associated with customers, which are, as a result, considered as citizens by health services.
• Big Data Advertisement on the Web for Opportunity Funding. In this context, vertical applications consider the issues related to using big data analytics (also combined with artificial intelligence algorithms) to support advanced Web marketing in various domains (work, investments, etc.). This represents a very exciting challenge for the next-generation community, as it is encouraged by already widely-adopted technologies and platforms for the Web (e.g., Google, Amazon, Alibaba, etc.) and for social networks (e.g., Facebook, Instagram, LinkedIn, etc.), which give providers and vendors the amenity for collecting and storing immense amounts of big data repositories representing user profiles, user preferences, user goals, etc. In addition to these big data repositories, artificial intelligence algorithms are performed in order to discover the best sub-optimal solution in a very large set of Web application cases, such as buying, investing, dating, etc.

As we demonstrate through the paper, the issue of big data management and analytics is playing a critical role in the actual research community. The SeDaSOMA framework’s proposal is a direct consequence of this clear trend. SeDaSOMA includes many research innovations, among which the serendipitous paradigm is the major one.

As regards future developments, we envision that critical big data applications will increasingly teach us what the novel requirements and features to capture will be, as was the situation in other relevant cases (e.g., COVID-19 pandemic management and analytics) (e.g., [45]). On the other hand, foundational and methodological aspects of big data management and analytics still need to establish themselves largely due to the fact that, on the application side, the evolution of big data management and analytics has been incredibly fast during the last few years.

All these considerations confirm for us the relevance of the research area we investigate and, to this end, the milestone represented by the SeDaSOMA proposal.

3.2. SeDaSOMA Implementation

As regards the specific implementation of the SeDaSOMA framework, we employ well-understood big data technologies.

Table 1. SeDaSOMA technologies.

Layers	Technology
Big Data Source Layer	Oracle NoSQL Database, MongoDB, Neo4J, Apache Streaming
Big Data Repository and Provisioning Layer	Apache Hadoop
Big Data Analytics Layer	Apache Spark
Big Data Application Layer	Java, C#, J#, Python

reports, for each of the main layers of SeDaSOMA (see Figure 1), the technology adopted.

At the Big Data Source Layer, there are the following: (i) Oracle NoSQL Database provides support for key–value data; (ii) MongoDB provides support for document-shaped big data (e.g., JSON); (iii) Neo4J provides support for big graph data; (iv) Apache Streaming provides support for streaming big data. The Big Data Repository and Provisioning Layer is completely implemented on top of Apache Hadoop. Apache Spark is the fundamental technology for the Big Data Analytics Layer, with its well-known task-focused libraries such as Spark SQL for SQL and structured data, MLlib for Machine Learning, Spark Streaming for stream processing, and GraphX for graph analytics. Finally, common high-level programming languages, such as Java, C#, J#, and Python, form the basis of the Big Data Application Layer.

In more detail, our proposed framework SeDaSOMA is integrated as a component of the Big Data Management and Analytics System architecture, as shown in Figure 2. This architecture consists of the following main components.

• Big Data Sources. In contemporary data environments, enterprises utilize a combination of cloud and on-premise resources to access and manage diverse big data sources. Cloud platforms offer scalability and accessibility, allowing enterprises to store and process large volumes of data efficiently. Within these cloud environments, enterprises integrate various data sources, including social media platforms, which provide valuable insights into consumer behavior and market trends through user-generated content and engagement metrics. Additionally, enterprises leverage activity generated from their own digital channels, such as websites and mobile applications, to gain real-time insights into user interactions and preferences. On-premise infrastructure complements cloud resources by hosting proprietary archives of historical data and legacy systems, ensuring data continuity and regulatory compliance. Furthermore, enterprises tap into public data repositories and external sources to enrich their datasets with contextual information from governmental databases, open data initiatives, and third-party APIs. Regarding the handling the different types of big data at this level of the architecture, Oracle NoSQL Database can provide support for the following: (i) key–value data; (ii) MongoDB for document-shaped big data (e.g., JSON); (iii) Neo4J for big graph data; (iv) Apache Streaming for big stream data. This hybrid approach to big data management enables enterprises to leverage a diverse array of sources to support decision-making processes and drive innovation.
• Big Data Archives. Within the architecture of big data management and analytics systems, the big data archive component plays a crucial role in facilitating the storage, preservation, and accessibility of vast volumes of data. Serving as a repository for historical and infrequently accessed data, the big data archive component ensures the long-term retention of valuable information while optimizing storage resources and cost-effectiveness. Leveraging scalable storage technologies and efficient data compression techniques, this component accommodates the ever-growing influx of data generated by diverse sources. Moreover, the big data archive component incorporates robust data governance [46] and security mechanisms to safeguard sensitive information and adhere to regulatory compliance requirements. By seamlessly integrating with other components of the big data ecosystem, such as data processing and analytics modules, the archive component enables the efficient retrieval and analysis of archived data, thereby facilitating informed decision-making and fostering innovation in data-driven enterprises using.
• Transactional Systems. This component assumes a pivotal role in enabling real-time processing and management of high-volume transactional data streams. This component encompasses sophisticated data processing frameworks and distributed transaction processing engines designed to handle massive concurrent transactions with low latency and high throughput. Leveraging scalable and fault-tolerant architectures, the big data transactional system ensures the reliability and integrity of transactional data in dynamic and distributed environments. Furthermore, it integrates with various data sources and downstream analytics modules to enable continuous data ingestion, processing, and analysis, thus supporting timely decision-making and operational intelligence in enterprise settings. Additionally, the big data transactional system component incorporates advanced transaction management functionalities, including distributed locking mechanisms, transaction isolation levels, and conflict resolution strategies, to maintain data consistency and concurrency control across distributed computing nodes.
• Big Data Engine. This comprises the Hadoop Distributed File System (HDFS) and MapReduce, therefore constituting a fundamental infrastructure for scalable and parallel processing of large datasets. Hadoop HDFS serves as a distributed file system designed to store and replicate data across a cluster of commodity hardware, ensuring fault tolerance and high availability. Concurrently, MapReduce provides a programming model and runtime environment for distributed data processing, enabling efficient parallel execution of computational tasks across distributed computing nodes. Together, Hadoop HDFS and MapReduce form the backbone of big data processing frameworks, facilitating the distributed storage and processing of massive datasets in a fault-tolerant and cost-effective manner. This component plays a pivotal role in supporting diverse analytics workflows, ranging from batch processing and data warehousing to real-time stream processing and machine learning, thereby enabling enterprises to derive actionable insights and drive innovation through data-driven decision-making.
• Operational Data Store (ODS). This component assumes a critical role as an intermediary storage layer facilitating real-time access and integration of heterogeneous data sources. The ODS acts as a centralized repository for ingesting, cleansing, and harmonizing streaming and batch data from diverse operational systems, sensors, and external sources. By providing a unified view of operational data in near real-time, the ODS enables organizations to make informed decisions, monitor performance, and respond promptly to evolving business needs. Leveraging scalable distributed architectures and advanced data processing techniques, the ODS ensures data consistency, reliability, and timeliness, thus serving as a foundational component for downstream analytics, reporting, and decision support applications. Moreover, the ODS fosters interoperability and data sharing across disparate systems and departments, driving collaboration and innovation in data-driven enterprises.
• Big Data Analytics (SeDaSOMA). Our proposed framework assumes a central role, employing a suite of advanced tools and frameworks to extract actionable insights from vast and diverse datasets. Utilizing Spark SQL for SQL-based querying and processing of structured data, this component enables efficient and scalable data manipulation and analysis, facilitating exploratory data analysis, data visualization, and ad-hoc querying tasks. Complementing Spark SQL, MLlib provides a comprehensive library of machine learning algorithms and utilities, empowering users to build and deploy scalable machine learning models for classification, regression, clustering, and collaborative filtering tasks. Furthermore, Spark Streaming facilitates real-time stream processing and analysis of continuous data streams, enabling organizations to derive timely insights and respond promptly to evolving trends and events. Additionally, GraphX offers a powerful framework for graph analytics, supporting graph processing algorithms and graph-based analytics applications such as social network analysis, recommendation systems, and fraud detection. Together, these components form a robust and versatile analytics ecosystem, empowering organizations to unlock the full potential of big data and drive innovation through data-driven decision-making processes.

4. Discussion

The natural humus for big data management and analytical chores is distributed environments. Cloud computing systems are among the most important, and they have even been sparked by current technological developments that have greatly improved the ICT sector right now (e.g., [47]). Real-world Cloud-based applications, like smart cities, intelligent transportation systems, marketplace tools, and so on, are increasingly creating new challenges for big data study, which advances the science.

Big data management and big data analytics in remote systems effectively merge inside a common, unifying framework whose primary concerns and challenges call for common solutions to universally handle the irksome problem of managing and supporting knowledge discovery from huge amounts of data.

Here, three top-class topics have thus emerged in actual studies:

• distributed big data management and analytics: complex methodologies;
• privacy-preservation methodologies for distributed big data management and analytics;
• distributed uncertain and imprecise big data management and analytics.

It should be highlighted that all of these themes not only pose pertinent theoretical challenges but also serve as reminders of substantial practical achievements in actual Cloud-based systems and applications. Consider, for example, the glaring example offered by genuine bio-informatics systems (e.g., [48]), where such concerns assume a significant role, to become convinced of this. To give an example in this specific context, SeDaSOMA can be successfully exploited to represent, process, and mine genome datasets, which are usually created within the context of major collaborations via comprehensive overshadowed data compilations generated by individual researchers in their respective laboratories, and shared upon publications. For instance, Array Express, a repository of publicly accessible gene expression data, houses over 1.3 million genome-wide assays derived from 45,000+ experiments. More precisely, what can be mined? Unsupervised techniques are employed to reveal the intrinsic organizational framework of the data, such as identifying patterns in gene expression related to cancers. These methodologies commonly detect prominent recurring characteristics within the data and may be influenced by various confounding factors. For instance, Principal Component Analysis (PCA) is a type of unsupervised method used to unveil hidden data features that deliver the most substantial information signal.

These important subjects are currently having a significant impact on the research community and will continue to play a significant part in both current and upcoming research endeavors. Following this unambiguous proof, we present the state-of-the-art on the referenced topics in this Section, along with ideas for directing additional field research.

4.1. Distributed Big Data Management and Analytics: Complex Methodologies

Numerous big data management and analytics systems, some of which have achieved excellence (e.g., [49,50,51]), have been proposed over time in the research literature. The next stage is moving toward advanced big data management and analytics in distributed contexts by looking into particular and brand-new research subjects (like those discussed in [52]). These topics first focus on effectively and efficiently supporting big data management and analytics tasks over challenging big data types, such as big social data (e.g., [53]), big multidimensional data (e.g., [54]), big graph data (e.g., [55]), big healthcare data (e.g., [56]), key–value stores (e.g., [57]), and so forth.

At the same time, designing innovative big data analytics tools that can extend the true capabilities of the most advanced methods is another hot topic of the proposal. Innovative big data analytics tools, such as multidimensional analysis (e.g., [58]), process mining analytics, reinforcement learning of big data analytics (e.g., [59]), etc., really require a new “call to arms” in the context of the literature. This has advanced today, with many promising scientific results.

Therefore, in line with these key research trends, the following are possible guidelines for next-generation research in the context of advanced big data management and analysis in a distributed environment:

• analysis of authoritative proposals in the investigated scientific area;
• selection of reference complex big data types;
• integration of complex big data repositories in specific Cloud data stores, such as NoSQL data layers;
• definition of novel big data management tasks on top of these data stores;
• project and implementation of big data management tasks;
• definition of novel big data analytics tools on top of these data stores;
• project and implementation of big data analytics tools.

4.2. Privacy-Preservation Methodologies for Distributed Big Data Management and Analytics

Like for other principal research topics, distributed big data management and analytics privacy-preservation methodologies (e.g., [60,61]) play a critical role, especially regarding the wide range of big data application scenarios that are emerging recently, which vary from social networks to bio-informatics, from sensors networks to web recommendation tools, from e-science systems to e-government systems, and so forth. The protection of the privacy of sensitive information can be referred to as enabling technology in all these applicative settings, whether for personal data (e.g., [62,63,64]) or aggregate data (e.g., [65,66,67]). Blockchain technology (e.g., [68,69]) is one of the significantly emerging topics in this context.

On the other hand, the process of privacy-preserving big data management and analytics is strongly related to the big data security research area (e.g., [70]), which investigates the means of accessing and managing big data repositories in a secure manner. However, there remains a problem that needs to be assessed, which is combining the privacy and security of big data in distributed environments. It is one of the main future research directions in big data.

The research community has already provided a large number of pieces of literature on privacy-preserving big data management and analytics in distributed environments in response to the growing interest in this topic, which has evolved in recent years (e.g., [62,71]). This shows how developed the subject is. However, it is certain that theoretical tools for facilitating big data management and analytics in distributed systems while preserving privacy constitute an exciting topic of study that has to be investigated in future works in this context. In this regard, an interesting research topic is to further extend well-consolidated theoretical models for privacy-preserving OLAP (e.g., [72]) to developing technologies like differential privacy (e.g., [73]). This paradigm should be extended even further to more broadly applicable privacy-preserving big data publishing problems (e.g., [71]). The combination of these problems with cutting-edge, sophisticated machine learning tools, like tensor-based big data analytics (e.g., [74]), represents a thriving field of research with excellent results in terms of both theoretical contributions and real-world applications. However, in terms of big data analytics proper, the problem of providing long-running big data analytics query processing in distributed environments (e.g., [75]), for instance, Cloud stores (e.g., [76]), in a privacy-preserving manner, represents another interesting line of research for the studied area. The primary challenge here is figuring out how to integrate the singleton query’s (e.g., OLAP query [77]) privacy preservation with the big data analytics task’s overall distributed privacy preservation, in order to build even more powerful and more expressive methodologies.

In the following, we provide some promising suggestions for future works in the field of privacy-preserving distributed big data management and analytics:

• sophisticated recommendation analytics for large data management and analytics that guarantee privacy in a distributed environment;
• identify case studies for privacy-preserving big data management and analysis in distributed environments (e.g., IoT, social networks, Cloud storage, etc.);
• specification of big data analytics and management tools/processes to be handled in distributed environments (e.g., OLAP, data publication, analytics-based tensor, long-term big data analytics, etc.);
• develop novel tools for privacy-preserving big data management and analysis in distributed environments, based on differential privacy theory, for example;
• design, implement, and test privacy-preserving big data management and analysis algorithms in a distributed environment;
• create and deploy benchmark case studies to comprehensively assess privacy-cyber-secure big data management and analysis in distributed environments;
• create and implement tuning solutions for predefined case studies.

4.3. Distributed Uncertain and Imprecise Big Data Management and Analytics

Among the issues and disadvantages regarding big data is that they are unpredictable and imprecise (e.g., [78,79]). In the case of sensor networks, which are some of the most popular sources of big data (e.g., [80,81]), Sensor-generated big stream data (e.g., [82,83]) are inherently ambiguous and inaccurate, which is similar to what occurs, for instance, with environmental sensor networks. In fact, there is usually some uncertainty in the monitored characteristic parameters such as temperature, pressure, humidity, and so forth.

This clear evidence has led to the recent emergence of approximate big data management and analytics models, methodologies, and approaches (e.g., [84,85]) as one of the top issues in big data research. Basically, big data management and analytics over (uncertain and imprecise) big data repositories are supported by approximation approaches and algorithms that are devised and built. Big data that are inaccurate and uncertain is usually modeled using probabilistic data models (e.g., [86]). It is clear that the aim of these techniques and algorithms is to assist with the basic procedures that form the basis for the development of more sophisticated approximation of big data management and analytics techniques. We identify approximate query answering algorithms (e.g., [87]), approximate search (e.g., [88]), approximation paradigms for supporting machine learning (e.g., [89]) clustering (e.g., [90]) tasks, and so forth, among the most intriguing proposals. Social Networks (such as [91,92]) offer a perfect example of a case study in which all of these techniques are applicable.

When all of these topics are combined, approximate big data management and analytics techniques and algorithms over imprecise and ambiguous big data repositories in distributed environments become one of the core contexts for next-generation big data research, which effectively merges significant technological advancements with solid theoretical frameworks.

In the active literature, there are many proposals focusing on approximate paradigms that support big data management and analytics over uncertain and imprecise big data repositories in distributed environments; some of these proposals, like [80,93,94], serve as important milestones for the scientific field, but problems and challenges persist. This research line aims to extend present achievements and solve these issues and difficulties by covering theoretical and technological gaps and enhancing the state of the art.

The challenge of computing OLAP aggregates over imprecise and unpredictable big data streams (e.g., [95]) is a well-consolidated situation in this regard, as these aggregates form the foundation of widely used big data analytics tools (e.g., [58]). With sampling-based techniques that have previously been shown to be successful in approximate query processing approaches, this paradigm readily scales to the more general challenge of providing scalable joins via flexible sample synopsis (e.g., [96]). Furthermore, these ideas can be easily extended to more probing scenarios, such as in [97], where sophisticated machine learning tasks leverage approximation calculations over imprecise and uncertain big data sources as baseline operations. However, query processing over uncertain big data also includes some types of questions that are worth exploring, such as top-k queries (e.g., [98]), RDF queries (e.g., [99]), pattern-matching on graphs (e.g., [100]), and so on. It should be highlighted that, as with big data in general, these concerns become especially interesting when applied to distributed environments. Here, it is necessary to genuinely extend classical approaches, which were primarily developed for centralized systems, to unique circumstances while maintaining a high level of creativity and innovation and taking into account the particular needs of distributed environments.

The following reports some potential directions for future research in the context of approximation methods and algorithms for big data management and analytics tasks over big data repositories in distributed environments:

• evaluation of state-of-the-art proposals in the context of approximate paradigms for supporting big data management and analytics in distributed environments;
• development of target uncertain and imprecise big data management and analytics scenarios in distributed environments for usage as case studies (e.g., Internet of Things, social networks, Cloud storage, and so forth);
• definition of target big data management and analytics tools and processes in distributed environments to be addressed, which are characterized by uncertainty and imprecision (e.g., OLAP over streaming big data, sensor networks, social networks, etc.);
• establishment of novel approximate big data analytics and management tools over imprecise and uncertain big data repositories in distributed environments, such as those based on probability theory;
• creation, execution, and evaluation of approximation-based big data management and analytics algorithms for unpredictable and inaccurate big data repositories in distributed environments;
• establishment and execution of benchmark case studies aimed at comprehensively evaluating approximate big data management and analytics techniques over ambiguous and imprecise big data repositories in distributed environments.

5. A Practical Implementation: The CORE-BCD-mAI Framework

In this Section, we introduce a practical implementation of SeDaSOMA, the CORE-BCD-mAI framework, namely “A COmposite Framework for REpresenting, Querying, and Analyzing Big Clinical Data by means of multidimensional AI Tools”. The proposed implemented framework considers the definition of innovative models, methodologies, techniques and algorithms, along with their experimental evaluation. These innovations are oriented to support the following: (i) representation; (ii) querying; (iii) analytics over big clinical data (e.g., [101]). This is achieved by exploiting meaningful multidimensional AI (Artificial Intelligence) tools such as the CORE-BCD-mAI framework. As will be made clear throughout this Section, CORE-BCD-mAI is a true realization of SeDaSOMA, which focuses on a specific case study (i.e., big clinical data).

To reach this ambitious goal, CORE-BCD-mAI truly introduces two main concepts: (i) big clinical data; (ii) multidimensional AI tools

Clinical data (i.e., medical tests, patient records, HVR (Heart Rate Variability)/ECG (ElectroCardioGram) data, omics, clinical trial data, etc.) is increasingly becoming a type of big data repository due to its intrinsic characteristics of volume, streaming, and diversity. This is reflected in the term big clinical data (see, for example, [102]). Among other characteristics, big clinical data is highly integrated with external data sources (e.g., census, social data, pharmacovigilance data, etc.), which further reinforces its (strong) variety. The latter characteristic opens the door to multi-level analytics methodologies that seek to analyze (large) clinical data at different levels of granularity (i.e., hospital level, regional level, national level) for supporting epidemiological studies oriented to health policy-oriented decision making.

The term “multidimensional AI tools” refers to a set of models, methods and analytics tools that are designed to support big data representativeness, management and analytics based on well-researched multidimensional frameworks. The most prominent multidimensional model among them is OLAP (OnLine Analytical Processing) (e.g., [16,103]). OLAP enables us to visualize and analyze data using happy multidimensional metaphors, such as dimensions, measures, levels, hierarchies, etc., by constructing highly intuitive multidimensional environments with analysis capabilities that exceed those of traditional SQL (Structured Query Language)-based analysis tools. In recent years, the research community has paid a lot of attention to the challenge of managing and analyzing big data through these paradigms (e.g., [13]).

Therefore, it is important to acknowledge that the representation, query, and analysis of large clinical data, such as patient records, HVR measurements and clinical trial data, through big data tools, and more specifically, multidimensional artificial intelligence tools, is a new research challenge with important results for the society.

It is no secret that ICT (Information and Communication Technologies) technologies can play a significant role in lowering the costs of national health care systems, as well as providing the necessary guidance for shaping future health care policies (see, for example, [104]).

Today, there is a lot of focus on how to effectively and efficiently manage and analyze large amounts of clinical data (e.g., [105]), especially because it has a significant impact on national health systems (and hence on society). In fact, there are several (large) national projects within this sector, such as in the USA (for example, [106]), Europe (for example, [107]) and South Korea (for example, [108]). However, the reality is that most of today’s Healthcare Information Systems (HIS) are still managed as (traditional) legacy IT systems, without taking advantage of the opportunities that big data technologies offer. In a sense, the primary objective of CORE-BCD-mAI is to fill this gap.

As big data technologies have been widely adopted, CORE-BCD-mAI exhibits a high degree of originality, in particular, because of the adaptation of multidimensional AI tools (which breaks the tradition of classical AI tools). On the other hand, CORE-BCD-mAI’s perspective is high because the proposed representation, query and analysis methodologies implemented within the framework can extend to other classes of data (e.g., social data or government data), just to name two well-known examples. CORE-BCD-mAI’s methodology focuses on designing and experimentally testing innovative AI models over synthetic and real-life big multidimensional clinical data sets.

Methodologies, Methods and Main Functionalities of CORE-BCD-mAI

The primary research claim of CORE-BCD-mAI is based on the argument that big data techniques—more specifically, multidimensional AI tools—are ideal for managing, processing, and assisting with analytics over big clinical data. This claim has also been the subject of other recent proposals in this area. The first step in achieving this goal and the central idea behind the CORE-BCD-mAI concept is the development of an appropriate multidimensional data model. In order to accomplish this, CORE-BCD-mAI presents the big multidimensional NoSQL data model, a particular data model that builds multidimensional structures on top of NoSQL data (such as OLAP data cubes; [103]), as the latter are acknowledged as the best option for representing big clinical data (e.g., [109]).

Classification, clustering, frequent itemset mining, association rule mining, and other classical data mining tools can all integrate with big data techniques. However, in order to benefit from rich 3V (e.g., [110]) data availability and the efficiency of consolidated mining approaches, these techniques must be parallelizable. Given the effectiveness and broad adoption of big data processing, administration, and mining tools and systems now in use, we can thus confidently define big data approaches as an enabling technology for CORE-BCD-mAI aims.

In order to support advanced big data analytics (including emerging big data visualization methodologies, e.g., [111,112]) for decision-making purposes, the primary goal of CORE-BCD-mAI is to develop models, methodologies, techniques, and algorithms based on multidimensional AI tools for the particular case of big clinical data to be processed, i.e., HVR measurements and statistics, ECG traces, patient record information, omics data, clinical guidelines, and so forth (including external data sources). This enables us to communicate novel best practices that will be taken into account in further research and to develop, execute, and experimentally evaluate tailored big data solutions that support CORE-BCD-mAI objectives.

Aiming to support the different stages of the previously mentioned data, information, and decisional processing flows, CORE-BCD-mAI also aims to define and implement a big multidimensional clinical data framework, known as the CORE-BCD-mAI framework. In order to achieve this, CORE-BCD-mAI uses well-known open-source big data processing technologies. This allows the framework to be used as widely as possible in both academic and industrial research communities, as well as to maximize the benefits of interoperability paradigms over freely available open-source data representation, management, and mining tools.

In Figure 3, the various stages of the research activities are emphasized, including the following: (i) Big Multidimensional Clinical Data Representation; (ii) Big Multidimensional Clinical Data Querying; (iii) Big Multidimensional Clinical Data Analytics; (iv) Big Multidimensional Clinical Data Visualization. This figure presents the overall data, information, and workflows of CORE-BCD-mAI and their interconnections.

Regarding particular implementation-related details, the proposed framework, which is fully based on SeDaSOMA (see Section 3.2), is built upon well-known Cloud Computing architectures (e.g., [113,114]), in which Hadoop (e.g., [115]) integrates various components (namely, big multidimensional clinical data representation, querying, analytics, and visualization)—mostly based on MapReduce (e.g., [116]), the most widely used big data processing paradigm. This strategy is based on fortunate situations where the Cloud Computing paradigm has already demonstrated its efficacy and efficiency in handling enormous volumes of big data.

Furthermore, the advantages of CORE-BCD-mAI over traditional data analytics tools for clinical data (e.g., [117]) should be highlighted. In contrast, CORE-BCD-mAI genuinely enables us to combine several types of data (e.g., clinical, patient, HVR/ECG, Omics, clinical trial, etc.) into a single integrated big data view. When paired with multidimensional modeling, this feature enables us to enhance the accuracy of the entire big data analytics process.

The different types of data sources are kept apart in the big data layer of CORE-BCD-mAI by contrasting traditional data warehousing approaches. This allows for the development of specialized machine artificial intelligence (mAI) tools for specific types of original datasets for correlation analysis and advanced knowledge discovery, possibly in comparison with knowledge findings derived from the integrated big data view (e.g., [118]).

This Section emphasizes that CORE-BCD-mAI incorporates multidimensional modeling and analytic methodologies at every system tier, not just in the final stage. This is, in fact, the proposal’s most significant addition. Multidimensional modeling and analytic techniques have, in fact, previously demonstrated their value in various contexts (e.g., [83]).

6. Conclusions and Future Work

This paper has focused on emerging trends in big data and introduced the main architecture of SeDaSOMA, a layered framework that supports Serendipitous, Data-as-a-Service-oriented, Open big data Management and Analytics. Our proposed framework is capable of supporting advanced big data management and analytics on different “hot” Cloud-aware application cases such as smart cities, social networks, sensor networks, intelligent transportation systems, etc. We further extended our contributions by providing some examples of Cloud-aware big data vertical applications of SeDaSOMA in particularly specific scenarios that capture a great deal of interest. In addition to this, we have also provided a critical overview of some emerging topics of big data research that are close to our work, by highlighting current state-of-the-art and future research efforts in the field. Finally, we have introduced a practical implementation of SeDaSOMA that focuses on big clinical data.

Our proposal is thus centered around the novel and emerging topic of open big data, which is an exciting paradigm for future years. Mostly, our proposal is concerned with paradigm aspects along with a reference framework and its architectural realization. We believe that these are relevant innovations in the field.

Future work is mainly oriented toward several assets: (i) design and implementation of a wide and comprehensive experimental campaign for assessing the performance of systems and tools developed according to the SeDaSOMA predicates; (ii) improving the overall proposal by advocating formal methods analysis, thus migrating toward a more conceived realization; (iii) exploiting SeDaSOMA as the foundational basis of future real-life research projects in order to stride-up the overall impact of our proposal. In addition to these practical goals, and looking at methodological perspectives, we aim at integrating our proposed framework with special big data features (e.g., [119]) such as the following: (i) visualization metaphors (e.g., [120]); (ii) flexible methodologies (e.g., [121,122,123]); (iii) uncertain management paradigms (e.g., [99,124,125]); (iv) intelligent data exchange approaches (e.g., [68,126,127]); (v) deep learning (e.g., [37]); (vi) federated learning (e.g., [128,129,130]).