From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape

Abstract

This comprehensive survey explored the evolving landscape of generative Artificial Intelligence (AI), with a specific focus on the recent technological breakthroughs and the gathering advancements toward possible Artificial General Intelligence (AGI). It critically examined the current state and future trajectory of generative AI, exploring how innovations in developing actionable and multimodal AI agents with the ability scale their “thinking” in solving complex reasoning tasks are reshaping research priorities and applications across various domains, while the survey also offers an impact analysis on the generative AI research taxonomy. This work has assessed the computational challenges, scalability, and real-world implications of these technologies while highlighting their potential in driving significant progress in fields like healthcare, finance, and education. Our study also addressed the emerging academic challenges posed by the proliferation of both AI-themed and AI-generated preprints, examining their impact on the peer-review process and scholarly communication. The study highlighted the importance of incorporating ethical and human-centric methods in AI development, ensuring alignment with societal norms and welfare, and outlined a strategy for future AI research that focuses on a balanced and conscientious use of generative AI as its capabilities continue to scale.

1. Introduction

Artificial Intelligence (AI) has advanced dramatically since Alan Turing’s “Imitation Game” [1], which first posed questions about machine cognition. Early computational theories [2,3] and the arrival of neural networks and machine learning [4,5,6] laid the groundwork for the deep learning era. These historical milestones set the stage for the next wave of progress, culminating in the creation of breakthrough Large Language Models (LLMs) such as OpenAI’s ChatGPT [7], Google’s Gemini [8] and Anthropic’s Claude [9], which have spurred fresh debates on AI consciousness and the consequences of advanced automation [10,11,12]. The most recent models extend their earlier capabilities beyond text-based interactions by handling inputs like images and audio while also demonstrating reasoning abilities that encompass multistep tasks, advanced mathematics, and code generation, therein repeatedly breaking benchmark performances and requiring more challenging ones to be devised [13].

Building on these strides, the field has seen further developments in both model architectures and operational strategies. Agentic AI equips LLMs with the ability to perform sophisticated tasks by leveraging external tools and APIs without constant human oversight [14]. Meanwhile, test-time compute or inference-time scaling increases efficiency by dynamically allocating computational resources at inferences based on the complexity of the query, as illustrated by OpenAI’s o-series models [7]. Prompting strategies, including Chain-of-Thought [15] and subsequent extensions like Tree-of-Thought [16] and Forest-of-Thought [17], provide technological foundations that enable the newest LLMs to follow a structured sequence of reasoning steps. These techniques support decision making via the generation long internal reasoning chains from which suitable solutions can be sampled and derived step by step, enabling a sophisticated multistage analysis of the possible solution space [18].

In parallel, Mixture of Experts (MoE) architectures have attracted attention for their capacity to distribute different computational tasks among specialized submodels [19]. This concept operates alongside the push toward omni-modal intelligence, where AI is expected to seamlessly process and integrate information across multiple sensory modalities. These approaches reflect a broader focus on versatile and adaptive systems that can handle a wide range of applications. The recently announced ’thinking models’ released by OpenAI and Google showcase previously unseen model capabilities in planning and reasoning-intensive tasks [20] that potentially point toward a future where AI systems may approximate general intelligence [21]. It is these particular and most consequential advancements that have produced models capable of deliberation, which have been linked to OpenAI’s technological breakthroughs speculated to be termed initially Q* and subsequently Strawberry [22].

Against this backdrop, our study investigates the transformative effect of modern generative AI innovations, including MoE architectures, Agentic AI, omni-modal frameworks and elastic compute strategies during inference. In particular, we focus on the most notable capabilities that the recent LLMs have acquired, namely, complex reasoning and signs of planning, which we will refer to as Q*-capabilities in the remainder of the manuscript. We examine how these developments are shaping research priorities, practical applications, and societal interactions, as well as the ethical, societal, and academic questions they raise. By presenting key insights and lessons learned, our findings aim to offer a roadmap for navigating this fast-evolving technological frontier.

1.1. Changing AI Research Popularity

As the field of LLMs continues to evolve, exemplified by recent breakthroughs, a multitude of studies have surfaced with the aim of charting future research paths, which have varied from identifying emerging trends to highlighting areas poised for swift progress. The dichotomy of established methods and early adoption is evident, with “hot topics” in LLM research increasingly shifting toward multimodal and omni-modal capabilities, as well as conversation-driven learning. The propagation of preprints has expedited knowledge sharing, but it also brings the risk of reduced academic scrutiny. Issues like inherent biases, noted by Retraction Watch, along with concerns about the misuse of generative AI, plagiarism, and forgery, present substantial hurdles [23,24,25]. The academic world, therefore, stands at an intersection, necessitating a unified drive to refine research directions in light of the fast-paced evolution of the field, which appears to be partly traced through the changing popularity of various research keywords over time. The release of commercial and powerful open-source generative models has been influential. As depicted in Figure 1, the rise and fall of certain keywords appear to have correlated with significant industry milestones, such as the release of the “Transformer” model in 2017 [26], the GPT model in 2018 [27], and the commercial ChatGPT-3.5 in December 2022. For instance, the spike in searches related to “Deep Learning” coincides with the breakthroughs in neural network applications, while the interest in “Natural Language Processing” surges as LLMs increasingly redefine what is possible in language understanding and generation. The enduring attention to “Ethics/Ethical” in AI research, despite some fluctuations, reflects the continuous and deep-rooted concern for the moral dimensions of AI, underscoring that ethical considerations are not merely a reactionary measure, but an integral and persistent dialogue within the AI discussion [28].

It is academically intriguing to postulate whether these trends signify a causal relationship, where technological advancements drive research focus, or if the burgeoning research itself propels technological development. This paper also explores the profound societal and economic impacts of AI advancements. We examine how AI technologies are reshaping various industries, altering employment landscapes, and influencing socio-economic structures. This analysis highlights both the opportunities and challenges posed by AI in the modern world, emphasizing its role in driving innovation and economic growth, while also considering the ethical implications and potential for societal disruption. Future studies could yield more definitive insights, yet the synchronous interplay between innovation and academic curiosity remains a hallmark of AI’s progress.

Meanwhile, the exponential increase in the number of preprints posted on arXiv under the Computer Science > Artificial Intelligence (cs.AI) category, as illustrated in Figure 2, appears to signify a paradigm shift in research dissemination within the AI community. While the rapid distribution of findings enables swift knowledge exchange, it also raises concerns regarding the validation of information. The surge in preprints may lead to the propagation of unvalidated or biased information, as these studies do not undergo the rigorous scrutiny and potential retraction typical of peer-reviewed publications [29,30]. This trend underlines the need for careful consideration and critique in the academic community, especially given the potential for such unvetted studies to be cited and their findings propagated.

1.2. Objectives

The impetus for this investigation has been the technological leaps forward demonstrated by the frontier LLMs both from OpenAI and Google, which have demonstrated the ability to “think” for extended durations before they respond. These models exhibit reasoning patterns akin to human problem solving by iteratively refining their problem-solving strategies, correcting errors, simplifying challenging steps, and adapting their approaches when current methods are insufficient. Notably, they exhibit advanced capabilities in complex reasoning, demonstrating their ability to decompose intricate problems into manageable subtasks and at least, emulate planning behaviors that were previously thought to be beyond the capabilities of existing LLM architectures [31,32]. Consequently, these noteworthy achievements call for an investigation into the current trends in generative AI research, which is critical for appraising the potential for obsolescence or insignificance in extant research themes while concurrently delving into burgeoning prospects within the rapidly transforming LLM panorama. This inquiry is reminiscent of the obsolete nature of encryption-centric or file-entropy-based ransomware detection methodologies, which have been eclipsed by the transition of ransomware collectives toward data theft strategies utilizing varied attack vectors, relegating contemporary studies on crypto-ransomware to the status of latecomers [33,34]. Advances in AI are anticipated to not only enhance the complex reasoning capabilities of LLMs and facilitate their integration into decision-making systems and IT infrastructure but also in the areas of omni-modality and robotics. LLM advancements to date have already heralded the obsolescence of conventional, statistics-driven natural language processing techniques in many domains [35], and others are have also been disrupted. Nonetheless, the perennial imperative for AI to align with human ethics and values persists as a fundamental tenet [36,37,38], and the conjectural Q-Star initiative offers an unprecedented opportunity to instigate discourse on how such advancements might reconfigure the LLM research topography. Our research methodology involved a structured literature search using key terms like ‘Large Language Models’ and ‘Generative AI’. To that end, this paper examines the potential impact of emerging technologies on research paths in AI, highlighting their role in opening new areas of exploration. It identifies several emerging research domains—MoE, multimodality, Agentic AI, adaptive models, and Artificial General Intelligence (AGI)—as pivotal in transforming generative AI research. The study presents a survey-style analysis to outline a research roadmap, focusing on both current and emerging trends in the field.

1.3. Survey Methodology and Scope

We use the term “survey” in the context of a “narrative literature review” or “scoping review” rather than a sociological study involving questionnaires. Our objective is to map current progress in generative AI, identify emerging research directions, and assess their relevance to academic and industrial domains.

Database Selection and Search Strategy: To capture the multidisciplinary nature of LLMs and generative AI, we consulted six major digital libraries commonly used in computer science and AI-related fields: IEEE Xplore, Scopus, ACM Digital Library, ScienceDirect, Web of Science, and ProQuest Central. We designed our queries by combining keywords such as “Generative AI”, “Large Language Models”, “Mixture of Experts”, “Gemini”, “Q-Star”, and “Artificial General Intelligence”. Given the rapid advancements since the publication of the Transformer model in 2017, our search primarily focused on literature from 2017 to 2024.

Inclusion and Exclusion Criteria: We screened articles based on the following criteria:

• Relevance to Generative AI Topics: We included works that significantly discuss LLMs, MoE architectures, multimodal learning, or AGI-oriented methods.
• Peer-Reviewed or Preprint: Priority was given to journal and conference publications; however, we also considered selective preprints (e.g., arXiv) if they substantially contributed new findings.
• Domain Focus: Papers centered on purely sociological, philosophical, or economic aspects without explicit generative AI components were excluded.

While we did not employ a strict systematic review protocol (e.g., PRISMA), our multidatabase approach helped ensure breadth and diversity in the sources included. Following initial screening, duplicates and clearly out-of-scope items were removed.

Quality Assessment and Synthesis: A pool of ∼600 candidate papers emerged from the initial queries. Through iterative reading and relevance checking, the final set was narrowed to 362 key references. In synthesizing the literature, we did the following:

• We organized papers by thematic clusters (e.g., Mixture-of-Experts Advances, Multimodal Extensions, or AGI Research Outlook).
• We cross-compared conflicting or convergent results to highlight consensus or outstanding debates.
• We derived a conceptual taxonomy to illustrate how new developments (e.g., Llama 2, Q-Star, etc.) potentially reshape the research landscape.

The resulting analysis formed the basis of our proposed classification framework, discussions regarding emerging directions, and reflection on ethical and technical challenges.

Limitations of Our Review: Our methodology did not involve a formal systematic review process nor explicit grading of evidence quality (e.g., GRADE). Additionally, with the volume of AI-related research expanding rapidly, certain recent arXiv papers might not be included if they appeared too late in our review cycle. Nevertheless, the review offers a comprehensive cross-section of key trends, drawing from prominent works and capturing the major research trajectories in generative AI.

The major contributions of this study are as follows:

• Detailed examination of the evolving landscape in generative AI, emphasizing the advancements and innovations in their complex reasoning capabilities and their wide-ranging implications within the AI domain.
• Analysis of the transformative effect of advanced generative AI systems on academic research, exploring how these developments are altering research methodologies, setting new trends, and potentially leading to the obsolescence of traditional approaches.
• Thorough assessment of the ethical, societal, and technical challenges arising from the integration of generative AI in academia, underscoring the crucial need for aligning these technologies with ethical norms, ensuring data privacy, and developing comprehensive governance frameworks.

The rest of this paper is organized as follows: Section 2 explores the historical development of generative AI. Section 3 presents a taxonomy of current generative AI research. Section 4 explores the MoE model architecture, its innovative features, and its impact on transformer-based language models. Section 5 discusses the enabling underlying architectures and reported capabilities of the latest frontier models, which embody those of the speculated Q* (In this paper, the term Q* serves as a symbolic representation that encompasses the recognized complex reasoning and early-stage planning advancements seen in frontier LLMs.) project. Section 6 discusses the projected capabilities of AGI. Section 7 examines the impact of recent advancements on the generative AI research taxonomy. Section 8 identifies emerging research priorities in generative AI. Section 10 discusses the academic challenges of the rapid surge of preprints in AI. The paper concludes in Section 11, summarizing the overall effects of these developments in generative AI.

2. Background: Evolution of Generative AI

The ascent of generative AI has been marked by significant milestones, with each new model paving the way for the next evolutionary leap. From single-purpose algorithms to LLMs like OpenAI’s ChatGPT and the latest multimodal systems, the AI landscape has been transformed, while countless other fields have been disrupted.

2.1. The Evolution of Language Models

Language models have undergone a transformative journey (Figure 3), evolving from rudimentary statistical methods to the complex neural network architectures that underpin today’s LLMs [39,40]. This evolution has been driven by a relentless quest for models that more accurately reflect the nuances of human language, as well as the desire to push the boundaries of what machines can understand and generate [39,40,41]. However, this rapid advancement has not been without its challenges. As language models have grown in capability, so too have the ethical and safety concerns surrounding their use, prompting a reevaluation of how these models are developed and the purposes for which they are employed [39,42,43,44,45].

2.1.1. Language Models as Precursors

The inception of language modeling can be traced to the statistical approaches of the late 1980s, which was a period marked by a transition from rule-based to machine learning algorithms in natural language processing (NLP) [46,47,48,49,50]. Early models, primarily n-gram based, calculated the probability of word sequences in a corpus, thus providing a rudimentary understanding of language structure [46]. Those models, simplistic yet groundbreaking, laid the groundwork for future advances in language understanding. With the increase in computational power, the late 1980s witnessed a revolution in NLP, pivoting toward statistical models capable of ‘soft’ probabilistic decisions, as opposed to the rigid, ‘handwritten’ rule-based systems that dominated early NLP systems [48]. IBM’s development of complicated statistical models throughout this period signified the growing importance and success of these approaches. In the subsequent decade, the popularity and applicability of statistical models surged, proving invaluable in managing the flourishing flow of digital text. The 1990s saw statistical methods firmly established in NLP research, with n-grams becoming instrumental in numerically capturing linguistic patterns. The introduction of Long Short-Term Memory (LSTM) networks in 1997 [51] and their application to voice and text processing a decade later [52,53,54] marked significant milestones, leading to the current era where neural network models represent the cutting edge of NLP research and development.

2.1.2. Large Language Models: Technical Advancement and Commercial Success

The advent of deep learning has revolutionized the field of NLP, leading to the development of early iterations of LLMs such as GPT and BERT, which laid the foundation for subsequent advancements. Codex, introduced by OpenAI [55], marked a significant step forward by fine-tuning language models for code generation, showcasing the versatility and domain-specific capabilities of LLMs. Notably, conversational agents like ChatGPT have further popularized the applications of these models. Recent innovations, including GPT-4o and the open-source LLaMA 3 family, have pushed the boundaries by integrating sophisticated techniques such as transformer architectures and advanced natural language understanding, illustrating the rapid evolution in this field [40]. These models represent a significant leap in NLP capabilities, leveraging vast computational resources and extensive datasets to achieve new heights in language understanding and generation [40,56]. ChatGPT has shown impressive conversational skills and contextual understanding with a broad spectrum of functional uses in many areas, as evidenced by its technical and commercial success, including rapid adoption by over 100 million users shortly after launch, which emphasizes a robust market demand for natural language AI and has catalyzed interdisciplinary research into its applications in sectors like education, healthcare, and commerce [35,56,57,58,59]. In education, ChatGPT offers innovative approaches to personalized learning and interactive teaching [57,60,61,62], while in commerce, it has revolutionized customer service and content creation [63,64]. The widespread use of ChatGPT, Google Bard/Gemini, Anthropic Claude, and similar commercial LLMs has reignited important debates in the field of AI, particularly concerning AI consciousness and safety, as its human-like interaction capabilities raise significant ethical questions and highlight the need for robust governance and safety measures in AI development [12,65,66,67]. Such influence appears to extend beyond its technical achievements, shaping cultural and societal discussions about the role and future of AI in our world.

The advancements in LLMs, including the development of models like GPT and BERT, have paved the way for the conceptualization of Q*-capabilities centered around complex reasoning and early-stage planning capabilities. Specifically, the scalable architecture, advanced reinforcement learning regimens, and test-time compute flexibilities that characterize these models are foundational to the newly acquired capabilities of frontier models like the GPT o-series and Gemini 2.0 models. Similarly, the emergence of multimodal systems capable of integrating text, images, audio, and video reflects an evolutionary path that that is being opened toward omni-modal models, combining the versatility of LLMs for more holistic and potentially fully sensory and embodied AI solutions that are able for the first time to acquire real-world models that current models lack [68] (

Table 1. Comparison of selected proprietary vs. open-source LLMs.

Model	Type	Developer	Key Features/Remarks
GPT-4o	Proprietary	OpenAI	State-of-the-art performance but limited transparency and restricted model access.
Gemini	Proprietary	Google	Multimodal focus, Agentic AI-capabilities; currently closed-source with advanced MMLU scores.
Claude	Proprietary	Anthropic	Emphasizes AI safety and interpretability; closed-source but widely used in industry.
GPT-NeoX [69]	Open-Source	EleutherAI	Community-driven LLM with openly released weights and configurations. Encourages reproducibility.
Llama 3 [70]	Open-Source (restricted)	Meta	Large-scale model with partial open license, fosters academic innovation with direct model access.
DeepSeek [21]	Open-Source	DeepSeek-AI	Large-scale MoE model, with multimodal versions DeepSeek-VL and DeepSeek-VL2, which integrate vision and language understanding.

).

2.1.3. Fine-Tuning, Hallucination Reduction, and Alignment in LLMs

The advancement of LLMs has underlined the significance of fine-tuning [71,72,73,74], hallucination reduction [75,76,77,78], and alignment [79,80,81,82,83]. These aspects are crucial in enhancing the functionality and reliability of LLMs. Fine-tuning, which involves adapting pretrained models to specific tasks, has seen significant progress: techniques like prompt-based and few-shot learning [84,85,86,87], alongside supervised fine-tuning on specialized datasets [71,88,89,90], have enhanced the adaptability of LLMs in various contexts, but challenges remain, particularly in bias mitigation and the generalization of models across diverse tasks [71,83,91]. Hallucination reduction is a persistent challenge in LLMs, which is characterized by the generation of confident but factually incorrect information [39]. Strategies such as confidence penalty regularization during fine-tuning have been implemented to mitigate overconfidence and improve accuracy [92,93,94]. Despite these efforts, the complexity of human language and the breadth of topics make completely eradicating hallucinations a daunting task, especially in culturally sensitive contexts [10,39]. Alignment, ensuring LLM outputs are congruent with human values and ethics, is an area of ongoing research. Innovative approaches, from constrained optimization [95,96,97,98,99] to different types of reward modeling [100,101,102,103], aim to embed human preferences within AI systems. While advancements in fine-tuning, hallucination reduction, and alignment have propelled LLMs forward, these areas still present considerable challenges. The complexity of aligning AI with the diverse spectrum of human ethics and the persistence of hallucinations, particularly on culturally sensitive topics, highlight the need for continued interdisciplinary research in the development and application of LLMs [10,104].

2.1.4. Mixture of Experts: A Paradigm Shift

The adoption of the MoE architecture in LLMs marks a critical evolution in AI technology. This innovative approach, exemplified by advanced models like Google’s Switch Transformer (https://huggingface.co/google/switch-c-2048; accessed on 10 January 2025) and MistralAI s Mixtral-8x7B (https://huggingface.co/mistralai/Mixtral-8x7B-v0.1; accessed on 10 January 2025), leverages multiple transformer-based expert modules for dynamic token routing, enhancing modeling efficiency and scalability. The primary advantage of the MoE lies in its ability to handle vast parameter scales, reducing memory footprint and computational costs significantly [105,106,107,108,109,110,111,112,113]. This is achieved through model parallelism across specialized experts, allowing for the training of models with trillions of parameters, and its specialization in handling diverse data distributions enhances its capability in few-shot learning and other complex tasks [106,107,114]. To illustrate the practicality of the MoE, consider its application in healthcare. For example, an MoE-based system could be used for personalized medicine, where different ‘expert’ modules specialize in various aspects of patient data analysis, including genomics, medical imaging, and electronic health records. This approach could significantly enhance diagnostic accuracy and treatment personalization. Similarly, in finance, MoE models can be deployed for risk assessment, where experts analyze distinct financial indicators, market trends, and regulatory compliance factors.

Despite its benefits, the MoE confronts challenges in dynamic routing complexity [115,116,117,118], expert imbalance [119,120,121,122], and probability dilution [123], and such technical hurdles demand sophisticated solutions to fully harness the MoE’s potential. Moreover, while the MoE may offer performance gains, it does not inherently solve ethical alignment issues in AI [19,124,125]. The complexity and specialization of MoE models can obscure the decision-making processes, complicating efforts to ensure ethical compliance and alignment with human values [124,126]. Although the paradigm shift to MoE signifies a major leap in LLM development, offering significant scalability and specialization advantages while ensuring the safety, ethical alignment, and transparency of these models remains a paramount concern. The MoE architecture, while technologically advanced, entails continued interdisciplinary research and governance to align AI with broader societal values and ethical standards.

2.2. Multimodal AI and the Future of Interaction

The advent of multimodal AI marks a transformative era in AI development, revolutionizing how machines interpret and interact with a diverse array of human sensory inputs and contextual data.

2.3. Advancing Multimodal AI: A Comparative Perspective of Frontier LLMs

The emergence of advanced multimodal AI models represents a transformative shift in artificial intelligence, enabling systems to process and integrate information across diverse modalities such as text, images, audio, and video [127,128,129]. Among the most notable contributions to this field are Google DeepMind’s Gemini 2.0, OpenAI’s GPT-4o, and Anthropic’s Claude 3.5 Sonnet. While these models share the overarching ambition of advancing multimodal AI, they embody distinct architectural strategies, feature sets, and optimization goals, reflecting diverse priorities in their development. Google DeepMind’s Gemini 2.0 is built on a sophisticated multimodal encoder–decoder framework [8,130,131], enabling seamless integration of text, images, audio, and video. This design enhances the model’s ability to perform comprehensive, context-aware reasoning, making it highly effective for real-world, dynamic interactions. Similarly, OpenAI’s GPT-4o employs an optimized multimodal architecture to prioritize real-time performance in response time for audio inputs, which facilitates latency-critical applications, and is mirrored by Anthropic’s Claude 3.5 Sonnet.

Agentic capabilities further complement and differentiate these models by extending their potential beyond passive interaction into active task execution. Gemini 2.0 emphasises agentic AI [132], with capabilities to autonomously plan, execute, and adapt actions in complex, real-world scenarios. This feature aligns Gemini 2.0 with applications requiring autonomy and proactive decision making. Claude 3.5 Sonnet similarly incorporates agentic functionalities, enabling it to interact directly with computer systems to automate tasks such as opening applications, managing workflows, and executing commands. In contrast, while GPT-4o does not explicitly highlight agentic features, its multimodal reasoning and rapid processing make it a versatile solution for real-time adaptations, where user-driven applications featuring interactivity and efficiency are paramount.

The architectural priorities and optimization goals of these models highlight their contrasting strengths and use cases. Gemini 2.0 is designed as a comprehensive solution for multimodal integration, excelling in dynamic, context-aware scenarios that demand advanced reasoning and adaptability. GPT-4o, with its focus on cost efficiency and rapid response times, caters to industries requiring scalable, real-time multimodal solutions. Meanwhile, Claude 3.5 Sonnet, with its Vision API and agentic task automation capabilities, targets applications demanding high-speed multimodal processing and operational precision. The collective advancements represented by these models mark a significant milestone in the development of multimodal AI [8,133] and pave the way for future universal or omni-modal models that are able to scale to a full complement of sensory inputs.

2.3.1. Gemini 2.0: Advancing Multimodal AI Capabilities

Gemini 2.0 [132], an advanced multimodal AI model developed by Google DeepMind, represents a significant leap in AI technology by enhancing the integration and processing of diverse data types, including text, images, audio, and video. This progression builds upon its predecessor’s foundation, introducing new features and improvements that set it apart in the realm of AI models.

The architecture of Gemini 2.0 incorporates a sophisticated multimodal encoder–decoder framework [8,130,131], enabling seamless understanding and generation across various modalities [8,127,128,129]. Key advancements include the following:

• Enhanced Multimodal Understanding: Gemini 2.0 exhibits improved capabilities in processing and reasoning across multiple modalities, allowing for more comprehensive and context-aware interactions.
• Native Image and Audio Generation: The model introduces native image generation and controllable text-to-speech functionalities, facilitating the creation of rich, multimodal content directly from textual inputs.
• Real-Time Multimodal Interactions: With the integration of the Multimodal Live API, Gemini 2.0 supports real-time audio and video streaming inputs, enabling dynamic and interactive AI experiences.
• Agentic Capabilities: Gemini 2.0 is designed for the agentic era, with the ability to plan, execute, and adapt actions autonomously, enhancing its utility in complex, real-world tasks.

These enhancements position Gemini 2.0 as a versatile and powerful AI model, capable of performing complex, multistep reasoning and providing more natural and efficient user interactions. Its deployment across various Google products and services highlights its significance in advancing AI applications and setting new benchmarks in the field [8,133].

2.3.2. Gemini: Redefining Benchmarks in Multimodality

Gemini, a pioneering multimodal conversational system, marks a significant shift in AI technology by surpassing traditional text-based LLMs like GPT-3 and even its multimodal counterpart, ChatGPT-4. Gemini’s architecture has been designed to incorporate the processing of diverse data types such as text, images, audio, and video, a feat facilitated by its unique multimodal encoder, cross-modal attention network, and multimodal decoder [8,127,128,129]. The architectural core of Gemini is its dual-encoder structure, with separate encoders for visual and textual data, enabling sophisticated multimodal contextualization [8,130,131]. This architecture is believed to surpass the capabilities of single-encoder systems, allowing Gemini to associate textual concepts with image regions and achieve a compositional understanding of scenes [8]. Furthermore, Gemini integrates structured knowledge and employs specialized training paradigms for cross-modal intelligence, setting new benchmarks in AI [8,133]. In [8], Google has claimed and demonstrated that Gemini distinguishes itself from ChatGPT-4 through several key features:

• Breadth of Modalities: Unlike ChatGPT-4, which primarily focuses on text, documents, images, and code, Gemini handles a wider range of modalities, including audio and video. This extensive range allows Gemini to tackle complex tasks and understand real-world contexts more effectively.
• Performance: Gemini Ultra excels in key multimodality benchmarks, notably in massive multitask language understanding (MMLU) which encompasses a diverse array of domains like science, law, and medicine, outperforming ChatGPT-4.
• Scalability and Accessibility: Gemini is available in three tailored versions—Ultra, Pro, and Nano—catering to a range of applications from data centers to on-device tasks and featuring a level of flexibility not yet seen in ChatGPT-4.
• Code Generation: Gemini’s proficiency in understanding and generating code across various programming languages is more advanced, offering practical applications beyond ChatGPT-4’s capabilities.
• Transparency and Explainability: A focus on explainability sets Gemini apart, as it provides justifications for its outputs, enhancing user trust and understanding of the AI’s reasoning process.

Despite these advancements, Gemini’s real-world performance in complex reasoning tasks that require the integration of commonsense knowledge across modalities remains to be thoroughly evaluated.

2.3.3. Technical Challenges in Multimodal Systems

The development of multimodal AI systems faces several technical hurdles, including creating robust and diverse datasets, managing scalability, and enhancing user trust and system interpretability [134,135,136]. Challenges like data skew and bias are prevalent due to data acquisition and annotation issues, which require effective dataset management by employing strategies such as data augmentation, active learning, and transfer learning [91,134,136,137]. A significant challenge is the computational demands of processing various data streams simultaneously, requiring powerful hardware and optimized model architectures for multiple encoders [138,139]. Advanced algorithms and multimodal attention mechanisms are needed to balance attention across different input media and resolve conflicts between modalities, especially when they provide contradictory information [139,140,141]. Scalability issues, due to the extensive computational resources needed, are exacerbated by limited high-performance hardware availability [142,143]. There is also a pressing need for calibrated multimodal encoders for compositional scene understanding and data integration [141]. Refining evaluation metrics for these systems is necessary to accurately assess performance in real-world tasks, calling for comprehensive datasets and unified benchmarks to enhance user trust and system interpretability through explainable AI in multimodal contexts. Addressing these challenges is vital for the advancement of multimodal AI systems, enabling seamless and intelligent interaction aligned with human expectations.

2.3.4. Multimodal AI: Beyond Text in Ethical and Social Contexts

The expansion of multimodal AI systems introduces both benefits and complex ethical and social challenges that extend beyond those faced by text-based AI. In commerce, multimodal AI can transform customer engagement by integrating visual, textual, and auditory data [144,145,146]. For autonomous vehicles, multimodality can enhance safety and navigation by synthesizing data from various sensors, including visual, radar, and Light Detection and Ranging (LIDAR) [146,147,148]. Still, DeepFake technology’s ability to generate convincingly realistic videos, audio, and images is a critical concern in multimodality, as it poses risks of misinformation and manipulation that significantly impact public opinion, political landscapes, and personal reputations, thereby compromising the authenticity of digital media and raising issues in social engineering and digital forensics where distinguishing genuine from AI-generated content becomes increasingly challenging [149,150]. Privacy concerns are amplified in multimodal AI due to its ability to process and correlate diverse data sources, potentially leading to intrusive surveillance and profiling, which raises questions about the consent and rights of individuals, especially when personal media are used without permission for AI training or content creation [134,151,152]. Moreover, multimodal AI can propagate and amplify biases and stereotypes across different modalities, and if unchecked, this can perpetuate discrimination and social inequities, making it imperative to address algorithmic bias effectively [153,154,155]. The ethical development of multimodal AI systems requires robust governance frameworks focusing on transparency, consent, data handling protocols, and public awareness, where ethical guidelines must evolve to address the unique challenges posed by these technologies, including setting standards for data usage and safeguarding against the nonconsensual exploitation of personal information [156,157]. Additionally, the development of AI literacy programs will be crucial in helping society understand and responsibly interact with multimodal AI technologies [134,156]. As the field progresses, interdisciplinary collaboration will be key in ensuring these systems are developed and deployed in a manner that aligns with societal values and ethical principles [134].

2.3.5. Technical Challenges and Limitations of Current LLMs

While LLMs have demonstrated exceptional capabilities, their transformer-based architectural design and operational paradigms posses significant limitations when applied to tasks requiring planning [31,68], forming dynamic models of the world [158], and conducting intricate, single-pass reasoning during inference [32]. These challenges, as well as those surrounding their lack of long-term memory and inability to accurately process and retain long-term dependencies [159], expose a gap between their remarkable generative capabilities and the complex demands of real-world decision making necessary for AGI.

A fundamental limitation of LLMs lies in their inability to form persistent, dynamic models of the world. Unlike humans or symbolic AI systems that can represent, simulate, and adapt to changing environments, LLMs rely exclusively on static patterns encoded in their training data [160]. This reliance prevents them from modeling and truly understanding causality, predicting long-term outcomes, or adapting their reasoning to real-world complexities. Additionally, planning, which is a marker of sophisticated intelligence and requires recourse to real-world models, therefore poses a significant challenge for current LLMs [31,68]. Effective planning requires creating and evaluating sequences of actions to achieve specific goals, often under uncertainty and with constraints. LLMs, however, operate as single forward-pass systems, inherently limiting their ability to engage in deliberative, sequential reasoning and causal inferences. Techniques like Chain-of-Thought (CoT) prompting and Tree-of-Thought (ToT) reasoning have enabled LLMs to emulate aspects of planning by breaking down problems into smaller reasoning steps. Yet, these methods rely heavily on prompt engineering and external scaffolding, as well as new approaches to reinforcement learning for automating them, thus lacking intrinsic mechanisms for true deliberative planning. As a result, LLMs generally struggle with tasks that demand strategic foresight, iterative problem-solving, or dynamic scenario evaluation.

Recent advancements, such as test-time compute scaling, have sought to address these challenges by dynamically allocating computational resources during inference. Models like OpenAI’s o1 and o3, as well as Gemini’s 2.0, exemplify this approach, enhancing task performance by allowing models to enter a “think” mode [132]. This entails generating a large set of competing possible chains of reasoning steps at inference time, which constitute a solution space from which a most suitable solution needs to be searched and found. However, while such a test-time compute scaling technique represents a significant step forward, it is resource-intensive and constrained by the existing architectures of LLMs, which can at best emulate planning and deliberative capabilities via extensive searches over possible solutions.

2.4. Advances in Complex Reasoning and Emerging Trends

The evolution of artificial intelligence continues to push the boundaries of complex reasoning and quasi-planning capabilities, as exemplified by recent advancements in models like OpenAI’s o1 and o3 and Google DeepMind’s Gemini 2.0. These models represent significant strides in integrating reinforcement learning, Chain-of-Thought (CoT) reasoning, and search-based techniques, addressing critical limitations in traditional LLM architectures. Here, we continue to use the term “Q*” as a symbolic representation to encapsulate the unification of structured reasoning, generative capabilities, and adaptive planning that these advancements aim to achieve.

From AlphaGo’s Legacy to the Symbolism of Q*

The transition from AlphaGo’s mastery of Go to the advanced reasoning capabilities symbolized by Q*-like techniques reflects a fundamental shift in AI research. AlphaGo showcased the potential of reinforcement learning coupled with Monte Carlo tree search to achieve superhuman performance in rule-based environments [161,162]. In contrast, today’s frontier models are extending these principles further to domains requiring complex, context-aware reasoning by leveraging reinforcement learning not just for optimization within bounded environments but also for broader, unstructured problem-solving tasks. In addition, memory augmentation techniques are being developed for LLMs, which are something they have lacked. Behrouz et al. [159] recently introduced a neural long-term memory module that augments transformer-based systems to enable them to integrate extensive historical context during inference without the computational overhead typically associated with large-context attention mechanisms. This innovation represents a significant step toward enhancing the reasoning and planning capabilities of AI systems by addressing limitations in handling long-range dependencies. The integration of these advancements, which includes bridging reinforcement learning and search-based algorithms during test-time compute, such as A*, can be encapsulated as realizations of Q* aspirations. Unlike AlphaGo’s focus on predefined rules, Q*-like methodologies aim to operate in open-ended, multimodal contexts and go further by introducing structured reasoning and adaptive planning, enabling AI systems to autonomously optimize decision paths, refine strategies through interaction, and seamlessly blend logical and generative capabilities.

2.5. Speculative Advances and Chronological Trends

In the dynamic landscape of AI, the emerging capabilities of the Q* techniques, blending LLMs, Q learning, and A* (A-Star algorithm), embody a significant leap forward. This section explores the evolutionary trajectory from game-centric AI systems to the broad applications anticipated with the ongoing technical breakthroughs of Q* approaches to mitigate the limitations of existing LLM architectures.

2.5.1. From AlphaGo’s Groundtruth to Q-Star’s Exploration

The journey from AlphaGo, a game-centric AI, to the Q*-like capabilities of new frontier models, represents a significant paradigm shift in AI. AlphaGo’s mastery in the game of Go highlighted the effectiveness of deep learning and tree search algorithms within well-defined rule-based environments, underscoring the potential of AI in complex strategy and decision making [161,162]. Q-Star, however, is speculated to move beyond these confines, aiming to amalgamate the strengths of reinforcement learning (as seen in AlphaGo), with the knowledge, NLG, creativity, and versatility of LLMs, as well as the strategic efficiency of pathfinding algorithms like A*. This blend, merging pathfinding algorithms and LLMs, could enable AI systems to transcend board game confines and, with Q-Star’s natural language processing, interact with human language, enabling nuanced interactions and marking a leap toward AI adept in both structured tasks and complex human-like communication and reasoning. Moreover, the incorporation of Q learning and A* algorithms would enable Q-Star to optimize decision paths and learn from its interactions, making it more adaptable and intelligent over time. The combination of these technologies could lead to AI that is not only more efficient in problem solving but also creative and insightful in its approach. This speculative advancement from the game-focused power of AlphaGo to the comprehensive potential of Q-Star illustrates the dynamic and ever-evolving nature of AI research, and it opens up possibilities for AI applications that are more integrated with human life and capable of handling a broader range of tasks with greater autonomy and sophistication.

2.5.2. Bridging Structured Learning with Creativity

The anticipated Q* project, blending Q learning and A* algorithms with the creativity of LLMs, embodies a groundbreaking step in AI, potentially surpassing recent innovations like Gemini. The fusion suggested in Q* points to an integration of structured, goal-oriented learning with generative, creative capabilities, forming a combination that could transcend the existing achievements of Gemini. While Gemini represents a significant leap in multimodal AI, combining various forms of data inputs such as text, images, audio, and video, Q* is speculated to bring a more profound integration of creative reasoning and structured problem solving. This would be achieved by merging the precision and efficiency of algorithms like A* with the learning adaptability of Q learning, as well as the complex understanding of human language and context offered by LLMs. Such an integration could enable AI systems to not only process and analyze complex multimodal data but also to autonomously navigate through structured tasks while engaging in creative problem solving and knowledge generation, mirroring the multifaceted nature of human cognition. The implications of this potential advancement are vast, suggesting applications that span beyond the capabilities of current multimodal systems like Gemini. By aligning the deterministic aspects of traditional AI algorithms with the creative and generative potential of LLMs, Q* could offer a more holistic approach to AI development. This could bridge the gap between the logical, rule-based processing of AI and the creative, abstract thinking characteristic of human intelligence. The anticipated unveiling of Q*, merging structured learning techniques and creative problem solving in a singular, advanced framework, holds the promise of not only extending but also significantly surpassing the multimodal capabilities of systems like Gemini, thus heralding another game-changing era in the domain of generative AI and showcasing its potential as a crucial development eagerly awaited in the ongoing evolution of AI.

3. The Current Generative AI Research Taxonomy

The field of Generative AI is evolving rapidly, which necessitates a comprehensive taxonomy that encompasses the breadth and depth of research within this domain. Detailed in

Table 2. Comprehensive taxonomy of current generative AI and LLM research.

Domain	Subdomain	Key Focus	Description
Model Architecture	Transformer Models	Efficiency, Scalability	Optimizing network structures for faster processing and larger datasets.
	Recurrent Neural Networks	Sequence Processing	Handling sequences of data, like text, for improved contextual understanding.
	MoE	Specialization, Efficiency	Leveraging multiple expert modules for enhanced efficiency and task-specific performance.
	Multimodal Models	Sensory Integration	Integrating text, vision, and audio inputs for comprehensive understanding.
Training Techniques	Supervised Learning	Data Labeling, Accuracy	Using labeled datasets to train models for precise predictions.
	Unsupervised Learning	Pattern Discovery	Finding patterns and structures from unlabeled data.
	Reinforcement Learning	Adaptability, Optimization	Training models through feedback mechanisms for optimal decision making.
	Transfer Learning	Versatility, Generalization	Applying knowledge gained in one task to different but related tasks.
Application Domains	Natural Language Understanding	Comprehension, Contextualization	Enhancing the ability to understand and interpret human language in context.
	Natural Language Generation	Creativity, Coherence	Generating coherent and contextually relevant text responses.
	Conversational AI	Interaction, Naturalness	Developing systems for natural and contextually relevant human–computer conversations.
	Creative AI	Innovation, Artistic Generation	Generating creative content, including text, art, and music.
Compliance and Ethical Considerations	Bias Mitigation	Fairness, Representation	Addressing and reducing biases in AI outputs.
	Data Security	Data Protection, Confidentiality	Ensuring data confidentiality, integrity, and availability security in AI models and outputs.
	AI Ethics	Fairness, Accountability	Addressing ethical issues such as bias, fairness, and accountability in AI systems.
	Privacy Preservation	Privacy Compliance, Anonymization	Protecting data privacy in model training and outputs.
Advanced Learning	Self-supervised Learning	Autonomy, Efficiency	Utilizing unlabeled data for model training, enhancing learning efficiency.
	Meta-learning	Rapid Adaptation	Enabling AI models to quickly adapt to new tasks with minimal data.
	Fine Tuning	Domain-Specific Tuning, Personalization	Adapting models to specific domains or user preferences for enhanced relevance and accuracy.
	Human Value Alignment	Ethical Integration, Societal Alignment	Aligning AI outputs with human ethics and societal norms, ensuring decisions are ethically and socially responsible.
Emerging Trends	Multimodal Learning	Integration with Vision, Audio	Combining language models with other sensory data types for richer understanding, leading to fully omni-modal models.
	Interactive and Cooperative AI	Collaboration, Human–AI Interaction	Enhancing AI’s ability to work alongside humans in collaborative tasks.
	AGI Development	Holistic Understanding	Pursuing the development of AI systems with comprehensive, human-like understanding.
	AGI Containment	Safety Protocols, Control Mechanisms	Developing methods to contain and control AGI systems to prevent unintended consequences.
	’Thinking Models’	Test-Time Compute, Inference Efficiency, Advanced Reinforcement Learning	Enabling AI systems to dynamically allocate computational resources during inference, enhancing flexibility and performance in complex reasoning tasks achieved by learned reasoning via reinforcement learning.
	Agentic AI	Autonomy, Task Execution	Empowering AI systems with the ability to autonomously plan, execute various tools, and adapt actions in real-world environments, aligning with human intentions and goals.
	Test Time Optimization	Adaptation, Memory	Adaptation at inference time via model updates and improved memory and ability to handle long dependencies.

, this taxonomy categorizes the key areas of inquiry and innovation in generative AI and serves as a foundational framework to understand the current state of the field, guiding through the complexities of evolving model architectures, advanced training methodologies, diverse application domains, ethical implications, and the frontiers of emerging technologies.

3.1. Model Architectures

Generative AI model architectures have seen significant developments, with four key domains standing out:

• Transformer Models: Transformer models have significantly revolutionized the field of AI, especially in NLP, due to their higher efficiency and scalability [163,164,165]. They employ advanced attention mechanisms to achieve enhanced contextual processing, allowing for more subtle understanding and interaction [166,167,168]. These models have also made notable strides in computer vision, as evidenced by the development of vision transformers like EfficientViT [169,170] and YOLOv8 [171,172,173]. These innovations symbolize the extended capabilities of transformer models in areas such as object detection, offering not only improved performance but also increased computational efficiency.
• Recurrent Neural Networks (RNNs): RNNs excel in the realm of sequence modeling, making them particularly effective for tasks involving language and temporal data, as their architecture is specifically designed to process sequences of data, such as text, enabling them to capture the context and order of the input effectively [174,175,176,177,178]. This proficiency in handling sequential information renders them indispensable in applications that require a deep understanding of the temporal dynamics within data, such as natural language tasks and time series analysis [179,180]. RNNs’ ability to maintain a sense of continuity over sequences is a critical asset in the broader field of AI, especially in scenarios where context and historical data play crucial roles [181].
• MoE: MoE models can significantly enhance efficiency by deploying model parallelism across multiple specialized expert modules, which enables these models to leverage transformer-based modules for dynamic token routing and to scale to trillions of parameters, thereby reducing both memory footprint and computational costs [106,115,182,183,184,185]. MoE models stand out for their ability to divide computational loads among various experts, each specializing in different aspects of the data, which allows for handling vast scales of parameters more effectively, leading to a more efficient and specialized handling of complex tasks [106,186].
• Multimodal Models: Multimodal models, which integrate a variety of sensory inputs such as text, vision, and audio, are crucial in achieving a comprehensive understanding of complex datasets, making them particularly transformative in fields like medical imaging [8,134,136,187]. These models facilitate accurate and data-efficient analysis by employing multiview pipelines and cross-attention blocks [188,189]. This integration of diverse sensory inputs allows for a more nuanced and detailed interpretation of data, enhancing the model’s ability to accurately analyze and understand various types of information [190]. The combination of different data types, processed concurrently, enables these models to provide a holistic view, making them especially effective in applications that require a deep and multifaceted understanding of complex scenarios [134,190,191,192].

3.2. Training Techniques

The training of generative AI models leverages four key techniques, each contributing uniquely to the field:

• Supervised Learning: Supervised learning, a foundational approach in AI, uses labeled datasets to guide models towards accurate predictions, and it has been integral to various applications, including image recognition and NLP [193,194,195]. Recent advancements have focused on developing sophisticated loss functions and regularization techniques, aimed at enhancing the performance and generalization capabilities of supervised learning models, ensuring they remain robust and effective across a wide range of tasks and data types [196,197,198].
• Unsupervised Learning: Unsupervised learning is essential in AI for uncovering patterns within unlabeled data, which is a process central to tasks like feature learning and clustering [199,200]. This method has seen significant advancements with the introduction of autoencoders [201,202] and Generative Adversarial Networks (GANs) [203,204,205], which have notably expanded unsupervised learning’s applicability, enabling more sophisticated data generation and representation learning capabilities. Such innovations are crucial for understanding and leveraging the complex structures often inherent in unstructured datasets, highlighting the growing versatility and depth of unsupervised learning techniques.
• Reinforcement Learning: Reinforcement learning, characterized by its adaptability and optimization capabilities, has become increasingly vital in decision making and autonomous systems [206,207]. This training technique has undergone significant advancements, particularly with the development of Deep Q-Networks (DQNs) [208,209,210] and Proximal Policy Optimization (PPO) algorithms [211,212,213]. These enhancements have been crucial in improving the efficacy and applicability of reinforcement learning, especially in complex and dynamic environments. By optimizing decisions and policies through interactive feedback loops, reinforcement learning has established itself as a crucial tool for training AI systems in scenarios that demand a high degree of adaptability and precision in decision making [214,215].
• Transfer Learning: Transfer learning emphasizes versatility and efficiency in AI training, allowing models to apply knowledge acquired from one task to different yet related tasks, which significantly reduces the need for large labeled datasets [216,217]. Transfer learning, through the use of pretrained networks, streamlines the training process by allowing models to be efficiently fine-tuned for specific applications, thereby enhancing adaptability and performance across diverse tasks and proving particularly beneficial in scenarios where acquiring extensive labeled data is impractical or unfeasible [218,219].

3.3. Application Domains

The application domains of Generative AI are remarkably diverse and evolving, encompassing both established and emerging areas of research and application. These domains have been significantly influenced by recent advancements in AI technology and the expanding scope of AI applications.

• Natural Language Understanding (NLU): NLU is central to enhancing the comprehension and contextualization of human language in AI systems, and it involves key capabilities such as semantic analysis, named entity recognition, sentiment analysis, textual entailment, and machine reading comprehension [220,221,222,223]. Advances in NLU have been crucial in improving AI’s proficiency in interpreting and analyzing language across a spectrum of contexts, ranging from straightforward conversational exchanges to intricate textual data [220,222,223]. NLU is fundamental in applications like sentiment analysis, language translation, information extraction, and more [224,225]. Recent advancements have prominently featured large transformer-based models like BERT and GPT-3, which have significantly advanced the field by enabling a deeper and more complex understanding of language subtleties [226,227].
• Natural Language Generation (NLG): NLG emphasizes the training of models to generate coherent, contextually relevant, and creative text responses, which are critical components in chatbots, virtual assistants, and automated content creation tools [39,228,229,230]. NLG encompasses challenges such as topic modeling, discourse planning, concept-to-text generation, style transfer, and controllable text generation [39,231]. The recent surge in NLG capabilities, exemplified by advanced models like GPT-3, has significantly enhanced the sophistication and nuance of text generation, which enable AI systems to produce text that closely mirrors human writing styles, thereby broadening the scope and applicability of NLG in various interactive and creative contexts [57,61,232].
• Conversational AI: This subdomain is dedicated to developing AI systems capable of smooth, natural, and context-aware human–computer interactions by focusing on dialogue modeling, question answering, user intent recognition, and multiturn context tracking [233,234,235,236]. In finance and cybersecurity, AI’s predictive analytics have transformed risk assessment and fraud detection, leading to more secure and efficient operations [34,234]. The advancements in this area, demonstrated by large pretrained models like Meena (https://neptune.ai/blog/transformer-nlp-models-meena-lamda-chatbots; accessed on 10 January 2025) and BlenderBot (https://blenderbot.ai; accessed on 10 January 2025), have significantly enhanced the empathetic and responsive capabilities of AI interactions. These systems not only improve user engagement and satisfaction, but also maintain the flow of conversation over multiple turns, providing coherent, contextually relevant, and engaging experiences [237,238].
• Creative AI: This emerging subdomain spans across text, art, music, and more, pushing the boundaries of AI’s creative and innovative potential across various modalities including images, audio, and video by engaging in the generation of artistic content, encompassing applications in idea generation, storytelling, poetry, music composition, visual arts, and creative writing, which have resulted in commercial successes like MidJourney and DALL-E [239,240,241], as well as most recently with state-of-the-art video generation models, namely, OpenAI’s Sora [242] and Google’s Veo 2 [243]. The challenges in this field involve finding suitable data representations, algorithms, and evaluation metrics to effectively assess and foster creativity [241,244]. Creative AI serves not only as a tool for automating and enhancing artistic processes but also as a medium for exploring new forms of artistic expression, enabling the creation of novel and diverse creative outputs [241]. This domain represents a significant leap in AI’s capability to engage in and contribute to creative endeavors, redefining the intersection of technology and art, with Sora and Veo 2 representing significant breakthroughs in the field of high-definition video generation that convincingly simulates real-world physics.

3.4. Compliance and Ethical Considerations

As AI technologies rapidly evolve and become more integrated into various sectors, ethical considerations and legal compliance have become increasingly crucial, which require a focus on developing ‘Ethical AI Frameworks’—a new category in our taxonomy reflecting the trend toward responsible AI development in generative AI [28,245,246,247,248]. Such frameworks are crucial in ensuring AI systems are built with a core emphasis on ethical considerations, fairness, and transparency, as they address critical aspects such as bias mitigation for fairness, privacy and security concerns for data protection, and AI ethics for accountability; thus, responding to the evolving landscape where accountability in AI is of paramount importance [28,245]. The need for rigorous approaches to uphold ethical integrity and legal conformity has never been more pressing, reflecting the complexity and multifaceted challenges introduced by the adoption of these technologies [28].

• Bias Mitigation: Bias mitigation in AI systems is a critical endeavor to ensure fairness and representation, which involves not only balanced data collection to avoid skewed perspectives but also involves implementing algorithmic adjustments and regularization techniques to minimize biases [249,250]. Continuous monitoring and bias testing are essential to identify and address any biases that may emerge from AI’s predictive patterns [250,251]. A significant challenge in this area is dealing with intersectional biases [252,253,254] and understanding the causal interactions that may contribute to these biases [255,256,257,258].
• Data Security: In AI data security, key requirements and challenges include ensuring data confidentiality, adhering to consent norms, and safeguarding against vulnerabilities like membership inference attacks [259,260]. Compliance with stringent legal standards within applicable jurisdictions, such as the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA), is essential, necessitating purpose limitation and data minimization [261,262,263,264]. Additionally, issues of data sovereignty and copyright emphasize the need for robust encryption, access control, and continuous security assessments [265,266]. These efforts are critical for maintaining the integrity of AI systems and protecting user privacy in an evolving digital landscape.
• AI Ethics: The field of AI ethics focuses on fairness, accountability, and societal impact; addresses the surge in ethical challenges posed by AI’s increasing complexity and potential misalignment with human values; and requires ethical governance frameworks, multidisciplinary collaborations, and technological solutions [28,245,267,268]. Furthermore, AI ethics involves ensuring traceability, auditability, and transparency throughout the model development lifecycle, as well as employing practices such as algorithmic auditing, establishing ethics boards, and adhering to documentation standards and model cards [268,269]. However, the adoption of these initiatives remains uneven, highlighting the ongoing need for comprehensive and consistent ethical practices in AI development and deployment [245].
• Privacy Preservation: This domain focuses on maintaining data confidentiality and integrity, as well as employing strategies like anonymization and federated learning to minimize direct data exposure, especially when the rise of generative AI poses risks of user profiling [270,271]. Despite these efforts, challenges such as achieving true anonymity against correlation attacks highlight the complexities in effectively protecting against intrusive surveillance [272,273]. Ensuring compliance with privacy laws and implementing secure data handling practices are crucial in this context, demonstrating the continuous need for robust privacy preservation mechanisms.

3.5. Advanced Learning

Advanced learning techniques, including self-supervised learning, meta-learning, and fine-tuning, are at the forefront of AI research, enhancing the autonomy, efficiency, and versatility of AI models.

• Self-supervised Learning: This method emphasizes autonomous model training using unlabeled data, reducing manual labeling efforts and model biases [195,274,275]. It incorporates generative models like autoencoders and GANs for data distribution learning and original input reconstruction [276,277,278], as well as also includes contrastive methods such as SimCLR [279] and MoCo [280], which are designed to differentiate between positive and negative sample pairs. Further, it employs self-prediction strategies, inspired by NLP, using techniques like masking for input reconstruction, which are significantly enhanced by recent vision transformers developments [195,281,282]. This integration of varied methods highlights self-supervised learning’s role in advancing AI’s autonomous training capabilities.
• Meta-learning: Meta-learning, or ‘learning to learn’, centers on equipping AI models with the ability to rapidly adapt to new tasks and domains using limited data samples [283,284]. This technique involves mastering the optimization process and is critical in situations with limited data availability to ensure models can quickly adapt and perform across diverse tasks, which are essential capacities in the current data-driven landscape [285,286]. It focuses on few-shot generalization, enabling AI to handle a wide range of tasks with minimal data, underlining its importance in developing versatile and adaptable AI systems [286,287,288,289].
• Fine-tuning: This involves customizing pretrained models to specific domains or user preferences, enhancing accuracy and relevance for niche applications [71,290,291]. Its two primary approaches are end-to-end fine-tuning, which adjusts all weights of the encoder and classifier [292,293], and feature extraction fine-tuning, where the encoder weights are frozen to extract features for a downstream classifier [294,295,296]. This technique ensures that generative models are more effectively adapted to specific user needs or domain requirements, making them more versatile and applicable across various contexts.
• Human Value Alignment: This emerging aspect concentrates on harmonizing AI models with human ethics and values to ensure that their decisions and actions mirror societal norms and ethical standards, involving the integration of ethical decision-making processes and the adaptation of AI outputs to conform with human moral values [100,297,298]. This is increasingly important in scenarios where AI interacts closely with humans, such as in healthcare, finance, and using personal assistants, to ensure that AI systems make decisions that are not only technically sound, but also ethically and socially responsible, which means human value alignment is becoming crucial in developing AI systems that are trusted and accepted by society [100,104,299].

3.6. Emerging Trends

Emerging trends in generative AI research are shaping the future of technology and human interaction, and they indicate a dynamic shift towards more integrated, interactive, and intelligent AI systems, driving forward the boundaries of what is possible in the realm of AI. Key developments in this area include the following:

• Multimodal Learning: Multimodal learning in AI, a rapidly evolving subdomain, focuses on combining language understanding with computer vision and audio processing to achieve a richer, multisensory context awareness [135,300]. Recent developments like the Gemini model have set new benchmarks by demonstrating state-of-the-art performance in various multimodal tasks, including natural image, audio, and video understanding, as well as mathematical reasoning [8]. Gemini’s inherently multimodal design exemplifies the seamless integration and operation across different information types [8]. Despite the advancements, the field of multimodal learning still confronts ongoing challenges, such as refining the architectures to handle diverse data types more effectively [301,302], developing comprehensive datasets that accurately represent multifaceted information [301,303], and establishing benchmarks for evaluating the performance of these complex systems [13,304,305]. Omni-modal models represent a progression from multimodal systems, aiming to achieve universal sensory integration by enabling seamless processing across all conceivable modalities, including text, images, audio, video, and even tactile or environmental data. These models signify a shift toward unified architectures that transcend the limitations of modality-specific processing, enhancing generalizability and contextual reasoning by integrating diverse sensory inputs into a cohesive understanding. Recent research, such as OmniBench [306], has introduced benchmarks designed to evaluate models’ ability to interpret and reason across visual, acoustic, and textual inputs simultaneously. Studies [307] in this domain highlight the need for robust multimodal integration techniques and training strategies to enhance performance across diverse modalities, which address challenges in establishing holistic understanding and reasoning in real-world scenarios.
• Interactive and Cooperative AI: This subdomain aims to enhance the capabilities of AI models to collaborate effectively with humans in complex tasks [38,308]. This trend focuses on developing AI systems that can work alongside humans, thereby improving user experience and efficiency across various applications, including productivity and healthcare [309,310,311]. Core aspects of this subdomain involve advancing AI in areas such as explainability [312], understanding human intentions and behavior (theory of mind) [313,314], and scalable coordination between AI systems and humans, which involve a collaborative approach crucial in creating more intuitive and interactive AI systems capable of assisting and augmenting human capabilities in diverse contexts [38,315].
• AGI Development: AGI, representing the visionary goal of crafting AI systems that emulate the comprehensive and multifaceted aspects of human cognition, is a subdomain focused on developing AI with the capability for holistic understanding and complex reasoning that closely aligns with the depth and breadth of human cognitive abilities [67,316,317]. AGI is not just about replicating human intelligence but also involves crafting systems that can autonomously perform a variety of tasks, demonstrating adaptability and learning capabilities akin to those of humans [316,317]. The pursuit of AGI is a long-term aspiration, continually pushing the boundaries of AI research and development.
• AGI Containment: AGI safety and containment acknowledges the potential risks associated with highly advanced AI systems, focused on ensuring that these advanced systems are not only technically proficient but also ethically aligned with human values and societal norms [12,28,67]. As we progress towards developing superintelligent systems, it becomes crucial to establish rigorous safety protocols and control mechanisms [12]. Key areas of concern include mitigating representational biases, addressing distribution shifts, and correcting spurious correlations within AI models [12,318]. The objective is to prevent unintended societal consequences by aligning AI development with responsible and ethical standards.
• “Thinking” Models: Thinking models represent a transformative trend in generative AI, focusing on integrating test-time compute scaling that enables computational flexibility during inference. These approaches allow AI systems to dynamically allocate additional computational resources based on task complexity, enhancing reasoning capabilities for problems requiring step-by-step deliberation. By allocating extra computational resources for reasoning-intensive tasks, these models achieve greater adaptability and efficiency, bridging the gap between traditional static inference and the demands of real-time, complex decision making. The development of thinking models highlights the potential to create systems capable of iterative, context-aware reasoning, addressing limitations in single-pass architectures while paving the way for more robust AI applications.
• Agentic AI: Agentic AI represents a critical advancement in empowering AI systems with autonomy, enabling them to plan, execute, and adapt actions in dynamic, real-world environments [319]. These systems leverage advanced tooling capabilities, allowing direct interaction with external software and hardware to achieve predefined objectives. Agentic AI also incorporates decision making and actionability, where the AI autonomously determines the optimal sequence of steps to achieve a goal while adhering to constraints. The development of agentic AI aligns with human-centric design principles, ensuring these systems are ethically aligned and augment human capabilities in diverse fields ranging from healthcare to logistics. However, the rise of agentic AI necessitates addressing challenges such as value alignment, safety protocols, and mitigating unintended consequences in autonomous decision making [319].
• Test Time Optimization: The ability for LLMs to dynamically adjust model parameters at inference time is an important research direction for LLMs. Test Time Training (TTT) [320,321] addresses the limitations of static, pretrained models by enabling LLMs to update parameters for smaller models during inference, especially for task-specific requirements. This facilitates real-time learning, allowing models to refine their understanding of a task as new inputs are encountered. This shift toward more adaptable systems allows LLMs to learn from each unique context and improve responses dynamically while also addressing computational limitations associated with long sequences. Approaches like TTT also mitigate the problem of memorization by learning how to memorize, rather than simply memorizing training data, enhancing generalization. This adaptability is can be seen as being complemented by enhanced long-term neural memory [159], which stores and recalls past information, moving beyond the limitations of short context windows in standard attention. These solutions are composed of short-term memory (core attention), long-term memory that learns to memorize at test time, and persistent memory, which stores task-specific knowledge. These advancements are pivotalm as they enable more flexible, context-aware AI systems capable of improved accuracy, better generalization, and more dynamic reasoning in complex, real-world applications.

4. Innovative Horizon of MoE

Recent advances in LLMs, multimodal frameworks, and agentic AI have renewed interest inMo) architectures as a crucial approach for scaling model capacity. This section provides an updated examination of MoE’s core concepts (Figure 4), recent progress, and prospective research directions in light of the latest literature, including improved parallelization and integration with advanced AI paradigms such as thinking models (test-time compute) and multistep planning.

4.1. Core Concept and Structure

The MoE model architecture continues to emerge as a scalable and specialized framework in transformer-era LLMs, enabling networks to partition complex tasks into specialized “expert” pathways [125,322,323]. Instead of relying on fully dense layers, the MoE replaces certain blocks with multiple feedforward “experts”, and a learnable router mechanism assigns each input token to only a subset of these experts, thus enforcing computational sparsity [106,324].

• Sparse Routing: Because only a few experts are activated per token, the MoE scales model capacity significantly without linearly increasing compute. This “conditional computation” is particularly valuable for tasks where a large portion of the model parameters may be unnecessary for every input.
• Transformer Integration: Recent LLM research incorporates MoE layers interleaved with attention blocks to maintain the contextual strengths of transformers while boosting efficiency for large-scale training [186].

As a result, MoE-based systems exhibit enhanced specialization across diverse tasks and can scale to trillions of parameters. However, key challenges such as dynamic load balancing, training stability, and memory overhead remain focal points for the community [106,325]. Furthermore, synergy with agentic AI demands robust gating logic that can flexibly adapt to environment-driven queries.

4.2. Training and Inference Efficiency

Recent LLMs confirm that MoE architectures significantly reduce training overhead while boosting performance across multilingual, coding, and domain-specific tasks [322]. By activating only a fraction of experts per token, these sparse layers lower the overall floating-point operations (FLOPs) while preserving or even improving accuracy relative to equally sized dense models.

Training Cost and Scalability: Studies have shown that advanced MoE setups can achieve up to 3–5× speedups in pretraining relative to dense transformer baselines [186,326]. Tools like Lina [327] tackle all-to-all communications to mitigate routing overhead, enhancing large-scale training parallelization. Even so, fine-tuning continues to impose heavy memory requirements in many MoE designs, as all experts may reside in GPU VRAM, though new work on hierarchical or load-on-demand experts is attempting to reduce this cost [125,328].

Inference Efficiency: Efficient serving of MoE-based LLMs is facilitated by advanced parallelism modes (e.g., expert parallelism or model compression) [322]. Compression strategies and micro-batching significantly decrease latency, with some efforts achieving up to 7–8× speedups at inference time [106]. Dynamic gating also opens the possibility of test-time compute scaling: the network can allocate additional experts for complex inputs, aligning with the concept of “thinking models” that allocate more compute to reason-intensive queries.

4.3. Load Balancing and Router Optimization

The gating network, or router, is integral to MoE’s effectiveness and stability [325,329]. Recent works introduced load-balancing losses (such as “Z-loss”) that prevent over-reliance on a small subset of experts, reducing training instabilities [330]. Techniques like capacity-limited experts also ensure that no single expert becomes saturated, maintaining throughput efficiency.

Robust Routing in Adversarial Settings: As LLMs become more agentic and are deployed in diverse real-world scenarios, adversarial examples or unusual input distributions may trigger degenerate routing. Ongoing research investigates robust gating to preserve balanced workloads even under adversarial attacks [326]. The synergy between gating and Chain-of-Thought or multimodal inputs remains a frontier area, where dynamic allocation of cross-modal “expert” modules can adapt in real time.

4.4. Parallelism and Serving Techniques

Realizing efficient MoE deployment at scale involves bridging HPC techniques with sophisticated software frameworks:

• DeepSpeed-MoE [322] introduces three main parallelism strategies—data parallelism, expert parallelism, and tensor slicing—to manage immense models (e.g., 1–2 trillion parameters) without exploding memory or latency.
• Expert Ensemble Approaches: In certain large-scale LLM tasks, MoE experts can function similarly to an ensemble model, boosting performance across specialized tasks like code generation, multilingual translation [331,332,333], or domain adaptation.
• MoE Model Compression: Hybrid or partial expert load can further reduce runtime, enabling more cost-effective serving. This direction aligns with industry demands for on-device or edge-based LLM inference where VRAM is at a premium [325].

Consequently, state-of-the-art MoE solutions can yield sublinear scaling of compute versus total model parameters [125,186,322,325], effectively democratizing trillion-parameter LLM capabilities. However, continued refinements—particularly in scheduling, caching, and dynamic routing—are needed to deliver truly universal and resource-flexible solutions for real-time multimodal and agentic applications.

4.5. Future Directions and Applications

Building on these innovations, the MoE paradigm is rapidly expanding in scope. We outline the following key open questions:

• Sparse Fine-Tuning and Instruction Tuning: Future research may explore gating-based fine-tuning that updates only a small subset of experts, enhancing computational and memory efficiency while retaining model quality. Instruction tuning can also be integrated, with specialized experts employed for each “instruction domain”.
• Multimodal Integration: The MoE can unify specialized experts for text, image, audio, and even 3D data, offering a flexible “mixture-of-modalities” approach. This is especially relevant for agentic AI systems requiring simultaneous processing of multiple data types.
• Adaptive Test-Time Compute: Research on on-demand expert activation—akin to recent “Thinking Model” frameworks—could allow LLMs to allocate more experts for complex tasks and fewer for simpler queries, optimizing both cost and performance.
• Calibration and Safety: Because experts can specialize in sensitive domains (e.g., healthcare, finance, etc.), robust mechanisms for bias mitigation and alignment with human values remain vital [80].

These directions underscore the MoE’s potential to serve as the backbone for next-generation AI ecosystems, spanning from massive cloud-based inference to resource-constrained edge scenarios. Aligning with emerging agentic AI and multimodal “LLM 2.0” paradigms, the MoE’s sparse design promises to deliver both scale and specialization, without incurring the prohibitive costs typical of traditional dense architectures. Through continued research in gating optimization, parallelization, compression, and dynamic inference, the MoE stands poised to redefine the frontier of large-scale neural network design and deployment.

5. Capabilities of Q*-Enabled Models

In light of recent advances in LLMs discussed in the previous section, the following formulations illustrate plausible ways that modules for reasoning, memory, and external retrieval could be interconnected in an advanced AI system referred to as “$Q^{*}$” (Figure 5). These equations are intended as conceptual frameworks rather than rigorous proofs, highlighting how modern LLM-based systems combine external memory, step-by-step reasoning, and reinforcement signals to address complex tasks.

5.1. Enhanced General Intelligence

Recent architectures frequently merge large-scale parameterized models with external retrieval and Chain-of-Thought techniques [15,16]. We capture this synergy as

$$\mathrm{EGI}(Q\ast )=\underset{i=1}{\overset{n}{⨁}}\left[{\mathrm{LLM}}_i(·)\otimes {\mathrm{CoT}}_i(·)\oplus {\mathrm{Retr}}_i\right],$$

(1)

Therein, we have the following:

• $\mathrm{EGI}(·)$ denotes a generalized intelligence operator integrating multiple modules;
• ${\mathrm{LLM}}_i$ is a large-scale language model trained on broad corpora;
• ${\mathrm{CoT}}_i(·)$ stands for Chain-of-Thought prompting and reasoning [15];
• ${\mathrm{Retr}}_i$ indicates retrieval modules for external knowledge (e.g., a vector database);
• ⊗ and ⊕ indicate functional integrations at different processing stages (e.g., fusing retrieval results with step-by-step reasoning).

This formulation highlights the current trend of combining parameter-rich models with lightweight retrieval and reasoning components to tackle broad, open-ended tasks efficiently.

5.2. Adaptive Self-Learning and Exploration

Building upon progress in self-play, hierarchical reinforcement learning, and reflection loops [334], we consider the following:

$$\mathrm{ASLE}(Q\ast )=\mathrm{HRL}\left(\mathrm{PNN},\mathrm{Mem}\right)\times \mathrm{Reflex},$$

(2)

Therein, we have the following:

• $\mathrm{ASLE}(·)$ is an adaptive self-learning and exploration operator;
• $\mathrm{HRL}(·)$ represents hierarchical reinforcement learning, splitting learning into high-level and low-level policies;
• $\mathrm{PNN}$ is a policy neural network interfacing with the hierarchical controllers;
• $\mathrm{Mem}$ denotes a knowledge buffer or episodic memory for iterative policy updates [335];
• $\mathrm{Reflex}$ implements reflection-based self-critique steps [334].

This highlights how next-generation RL systems can iteratively refine both high-level goals and low-level execution, leveraging memory and self-reflection mechanisms for continual improvement.

5.3. Multilayered Language Understanding and Reasoning

Rather than emulating strict human-level metrics, contemporary work emphasizes layered reasoning involving factual, logical, and commonsense modalities [80]. A possible abstraction is the following:

$$\mathrm{MLLU}(Q\ast )=\sum _{m\in {\mathcal{M}}_{\mathrm{Reason}}}\left[\mathrm{LLM}\left({\mathrm{Knowledge}}_m\right)\oplus {\mathrm{Align}}_m\right],$$

(3)

Therein, we have the following:

• $\mathrm{MLLU}(·)$ denotes a multilayered language understanding operator;
• ${\mathcal{M}}_{\mathrm{Reason}}$ enumerates various modes of reasoning (expert logic, common sense, etc.);
• $\mathrm{LLM}\left({\mathrm{Knowledge}}_m\right)$ processes specialized or curated knowledge resources;
• ${\mathrm{Align}}_m$ is an alignment or value filter [80] ensuring compliance with domain or ethical constraints;
• ⊕ merges each reasoning layer’s output.

This structured approach reflects the current focus on multistage, alignment-aware pipelines in advanced language reasoning systems.

5.4. Context-Specific Common Sense and Tool Usage

The research on tool usage and external plugins extends “common sense” beyond internal reasoning [336]. We propose the following:

$$\mathrm{CSCU}(Q\ast )=\left(\mathrm{CS}\mathrm{Engine}\otimes \mathrm{ToolAPI}\right)\oplus \mathrm{WorldK},$$

(4)

Therein, we have the following:

• $\mathrm{CSCU}(·)$ captures context-specific common sense and tool usage;
• $\mathrm{CS}\mathrm{Engine}$ is a core symbolic or Chain-of-Thought logic unit for everyday reasoning;
• $\mathrm{ToolAPI}$ encapsulates callable external utilities (e.g., search engines, calculators, code interpreters, etc.);
• $\mathrm{WorldK}$ is a dynamic knowledge base or curated fact store;
• ⊗ and ⊕ denote flexible chaining of these resources on demand.

This formulation aligns with frameworks like ToolFormer and plugin-based systems, enabling flexible, context-driven resource usage.

5.5. Ethics, Verification, and Transparency

Finally, modern AI systems integrate interpretability, safety, and ethics checking directly into their generative workflows [337]. We define the following:

$$\mathrm{EVT}(Q\ast )=\mathrm{Verifier}\left(\mathrm{LLM}\right)\otimes \mathrm{Explainer}\otimes \mathrm{EthicsCheck},$$

(5)

Therein, we have the following:

• $\mathrm{EVT}(·)$ stands for ethics, verification, and transparency;
• $\mathrm{Verifier}$ includes factual consistency or theorem-proving tools;
• $\mathrm{Explainer}$ produces Chain-of-Thought or post hoc rationales for interpretability;
• $\mathrm{EthicsCheck}$ embeds societal or policy-based constraints.

This ensures that advanced generative models maintain accountability and transparency in parallel with their raw computational capabilities.

6. Projected Capabilities of AGI

Artificial General Intelligence (AGI) is increasingly discussed as a prospective leap in AI research, aiming to replicate (or at least approximate) the breadth of human cognitive abilities within computational systems [67,316,317]. Recent progress in large-scale language models, multimodal integration, and agentic AI suggest possible pathways toward more autonomous systems, although significant conceptual and technical barriers remain. Figure 6 offers a schematic view of how AGI might encompass autonomous learning, broad cognitive faculties, common sense reasoning, and holistic knowledge integration.

6.1. Revolution in Autonomous Learning

AGI research aspires to push beyond static, dataset-centric training regimes by introducing systems that can continuously learn from interactions and environmental feedback [67,316,317,338]. Methods such as Proximal Policy Optimization (PPO) or other reinforcement-based strategies allow these models to adapt their parameters dynamically, indicating a paradigm shift from “train-then-infer” cycles to ongoing self-improvement [211,339]. This approach aims to reduce reliance on frequent human-initiated retraining, making the system more robust and responsive to nonstationary or novel scenarios.

6.2. Broadening of Cognitive Abilities

By integrating multiple architectures—ranging from large transformer-based models (e.g., GPT, BERT, etc.) to multimodal pipelines—AGI could exhibit a more holistic intelligence akin to human cognition [316,340]. Recent progress in multistep reasoning, “Chain-of-Thought” prompting, and agentic AI frameworks suggests that future systems might autonomously coordinate specialized modules or “universal adapters” to assimilate diverse information types in real time [316,341]. This adaptability is already beginning to show promise in fields like healthcare, where advanced diagnostic models indicate how context-rich learning could transform medical analysis and treatment planning.

6.3. Elevating Understanding and Interaction

AGI is projected to reach deeper linguistic and socio-emotional comprehension, leveraging large-scale models with sophisticated inference and possibly multisensory integration [316,338,342]. Beyond textual or visual modalities, such systems may interpret pragmatic cues—tone, affect, or user intent—enabling complex, empathetic, and context-sensitive engagements. This expansion from purely symbolic or pattern-based processing toward contextually grounded interactions marks a critical step in building AI systems that can meaningfully collaborate with humans in scientific research, creative endeavors, and decision making.

6.4. Advanced Common Sense Reasoning

While current LLMs often exhibit “hallucinations” or lack robust world modeling, a future AGI might incorporate symbolic reasoning, probabilistic inference, and real-time environment data to achieve common sense understanding [316,343,344]. Such systems could navigate real-world tasks more effectively, bridging the gap between narrowly trained AI and human-like flexibility. Symbolic submodules or knowledge graphs may augment generative models, offering explicit reasoning chains that align more closely with how humans conceptualize cause–effect relationships and domain knowledge.

6.5. Holistic Integration of Knowledge

As AGI efforts progress, new methods for fusing diverse knowledge representations—textual, visual, and multimodal—are emerging [316,342]. Such comprehensive synthesis could allow AGI systems to handle complex, cross-disciplinary challenges: from climate change modeling to economic policy simulations. At the same time, formal verification techniques and advanced alignment protocols aim to ensure that AGI’s outputs are not only accurate but also ethically or legally compliant [28,67]. If realized, these capabilities could yield impactful solutions in fields ranging from environmental sustainability to large-scale social planning [317,340].

6.6. Challenges and Opportunities in AGI Development

Despite the optimism surrounding AGI, significant hurdles persist. Models still grapple with issues like representational bias, interpretability, and the high computational overhead of large-scale architectures [28,67]. Balancing these challenges requires robust data governance, more efficient resource usage, and governance frameworks that address societal concerns ranging from privacy to misuse [340]. Furthermore, it remains unclear whether scaling existing architectures or forging entirely new paradigms (e.g., brain-inspired spiking networks) will prove most effective in achieving AGI [67,345].

Experts caution that we should not conflate current LLM-based “sparks of intelligence” with the full spectrum of human-level cognition [316]. Indeed, the gap between practical AI systems and truly general intelligence remains considerable. Nonetheless, the trajectory of agentic AI, multimodal reasoning, and dynamic training points to a future where AI could expand its autonomy across various domains [28,316]. Realizing AGI’s transformative potential thus requires diligent research, transparent collaboration among stakeholders, and the development of ethical guardrails—ensuring responsible innovation as these capabilities evolve.

7. Impact Analysis on Generative AI Research Taxonomy

This section presents an updated analysis of how recent developments and insights in generative AI—including thinking models, agentic AI, and test time optimization—influence existing research domains across model architectures, training techniques, application areas, compliance/ethics, advanced learning paradigms, and emerging trends. We first outline the criteria used to categorize the shifting research landscape and then provide a revised overview of how each subdomain in our taxonomy is impacted.

7.1. Criteria for Impact Analysis

The rapidly evolving landscape of generative AI catalyzes transformative shifts across diverse research domains. To systematically evaluate the influence of these advancements, we have established a robust set of criteria detailed in Table 3. These criteria serve as analytical lenses to quantify and categorize the impact of generative AI innovations that are grounded in the dynamic interplay between technological progress and the evolving paradigms of research focus areas.

Our analytical framework employs a gradient scale ranging from emergent to obsolete, reflecting the extent to which areas of generative AI research are being reshaped. The categorization into five distinct classes—Emerging Direction, Requiring Redirection, Still Relevant, Likely to Become Redundant, and Inherently Unresolvable—allows for a nuanced assessment, acknowledging that not all research areas are uniformly affected. This multitiered approach is informed by historical patterns of technological disruption and the adaptability of scientific inquiry [346,347].

Table 3. Criteria for analyzing impact on generative AI research.

Symbol	Criteria	Score	Definition	Justification
↗	Emerging Direction	5	New research areas expected to arise as a direct consequence of AI advancements.	Emphasizes novel research domains emerging from AI breakthroughs [346,348].
↪	Requiring Redirection	4	Areas that need to shift focus or methodology to stay relevant with new AI developments.	Technological shifts necessitate re-evaluation and redirection in AI research [346,347].
↔	Still Relevant	3	Areas where the advancements have minimal or no impact, maintaining their current status and methodologies.	Observes the persistence of certain AI research areas despite technological advancements [347].
↘	Likely to Become Redundant	2	Areas that may lose relevance or become obsolete with the advent of new AI technologies.	Discusses rapid obsolescence in AI methodologies due to new technologies [349].
△	Inherently Unresolvable	1	Challenges that may remain unresolved due to complexities like subjective human perspectives and diverse cultural values.	Inherent difficulties in issues such as aligning AI with diverse human values and ethics [350,351].

Table 3. Criteria for analyzing impact on generative AI research.

Symbol	Criteria	Score	Definition	Justification
↗	Emerging Direction	5	New research areas expected to arise as a direct consequence of AI advancements.	Emphasizes novel research domains emerging from AI breakthroughs [346,348].
↪	Requiring Redirection	4	Areas that need to shift focus or methodology to stay relevant with new AI developments.	Technological shifts necessitate re-evaluation and redirection in AI research [346,347].
↔	Still Relevant	3	Areas where the advancements have minimal or no impact, maintaining their current status and methodologies.	Observes the persistence of certain AI research areas despite technological advancements [347].
↘	Likely to Become Redundant	2	Areas that may lose relevance or become obsolete with the advent of new AI technologies.	Discusses rapid obsolescence in AI methodologies due to new technologies [349].
△	Inherently Unresolvable	1	Challenges that may remain unresolved due to complexities like subjective human perspectives and diverse cultural values.	Inherent difficulties in issues such as aligning AI with diverse human values and ethics [350,351].

7.1.1. Emerging Direction (↗)

At the apex of our evaluative hierarchy, Emerging Direction encapsulates the advent of uncharted research vistas propelled by ongoing AI breakthroughs. These are new or rapidly growing research areas that are highly influenced by current innovations, such as advanced reasoning capabilities and agentic AI systems [346,348]. This category highlights novel subfields driven by cutting-edge technologies, signaling significant shifts in research focus and the potential for groundbreaking discoveries.

7.1.2. Requiring Redirection (↪)

Requiring Redirection pertains to established research areas that must adapt or pivot to remain relevant in the face of new AI developments. These areas are at an inflection point, necessitating a strategic overhaul of traditional methodologies to integrate emergent paradigms like thinking models, multimodality, or advanced LLMs [346,347]. For instance, the transition from rule-based expert systems to adaptive machine learning frameworks exemplifies the need for redirection in existing research domains.

7.1.3. Still Relevant (↔)

The Still Relevant classification affirms the resilience of select research domains that retain their core significance despite new technological advancements. These areas maintain their relevance by addressing persistent scientific inquiries or through their inherent flexibility to incorporate new techniques and best practices [347]. Longstanding topics, such as fundamental algorithmic improvements or theoretical foundations, fall under this category, demonstrating the enduring nature of certain research pursuits.

7.1.4. Likely to Become Redundant (↘)

Areas categorized as Likely to Become Redundant face the risk of obsolescence due to the emergence of more efficient or generalizable methods. These research domains may be overshadowed by powerful new LLM architectures or emergent frameworks that render previous approaches less effective or relevant [349]. This classification serves as a warning for researchers to anticipate potential reductions in relevance and consider strategic foresight and resource reallocation to prevent scientific stagnation.

7.1.5. Inherently Unresolvable (△)

Finally, Inherently Unresolvable challenges represent topics that remain incomplete or irreconcilable, often due to complex human or cultural factors. These enduring dilemmas within AI research, such as fully universal ethics, are deeply rooted in the diverse tapestry of human values and societal imperatives [350,351]. This category highlights the perpetual nature of certain challenges, highlighting the necessity for ongoing dialogue and interdisciplinary approaches to navigate these complex issues.

The comprehensive categorization provided by these criteria enables a structured and detailed assessment of how Generative AI innovations influence various research domains. By systematically applying these labels, researchers can identify trends, anticipate shifts, and strategically align their work with the evolving landscape of AI advancements.

7.2. Overview of Impact Analysis

This subsection provides an updated and detailed overview of the impact analysis carried out on the research taxonomy within the realm of generative AI. We expand on the existing paradigms of MoEs, multimodality, and AGI by incorporating the newly emerging frameworks of thinking models (involving test-time compute) and agentic AI. Our goal is to illustrate how these collective advancements influence multiple facets of generative AI research, spanning from model architectures and sophisticated training methodologies to ethical, compliance, and governance issues. By synthesizing both quantitative and qualitative assessments across different domains and subdomains of LLM research, we highlight how each area is compelled to adapt, remain steadfast, or potentially yield to more advanced approaches.

In carrying out this analysis, several factors were considered. First, we investigated whether newly introduced techniques and system architectures (e.g., sparse routing, test time optimization, Chain-of-Thought, and real-time memory augmentation) have triggered emerging directions in established subfields. Second, we asked if existing research directions must redirect their approaches or frameworks to align with the capabilities and demands of next-generation LLMs. Third, we observed the continued relevance of foundational methodologies that may remain robust despite the introduction of more sophisticated generative systems. Finally, the analysis reveals whether certain lines of research have become less vital in light of these advancements or if core ethical dilemmas remain inherently unresolvable, owing to the complex nature of societal, psychological, or philosophical challenges.

Table 4. Impact of MoE, multimodality, and AGI on generative AI research.

Domain	Subdomain	MoE	Multimodality	AGI	Overall Score
Model Architecture	Transformer Models	↪ (4)	↔ (3)	↪ (4)	11
	Recurrent Neural Networks	↘ (2)	↔ (3)	↘ (2)	7
	MoE	↔ (3)	↗ (5)	↪ (4)	12
	Multimodal Models	↗ (5)	↔ (3)	↗ (5)	13
Training Techniques	Supervised Learning	↪ (4)	↔ (3)	↘ (2)	9
	Unsupervised Learning	↪ (4)	↔ (3)	↪ (4)	11
	Reinforcement Learning	↔ (3)	↪ (4)	↗ (5)	12
	Transfer Learning	↔ (3)	↗ (5)	↪ (4)	12
Application Domains	Natural Language Understanding	↔ (3)	↔ (3)	↗ (5)	11
	Natural Language Generation	↔ (3)	↪ (4)	↗ (5)	12
	Conversational AI	↪ (4)	↗ (5)	↗ (5)	14
	Creative AI	↪ (4)	↗ (5)	↗ (5)	14
Compliance and Ethical Considerations	Bias Mitigation	↪ (4)	↪ (4)	↗ (5)	13
	Data Security	↔ (3)	↔ (3)	↔ (3)	9
	AI Ethics	↪ (4)	↪ (4)	△ (1)	9
	Privacy Preservation	↪ (4)	↪ (4)	↪ (4)	12
Advanced Learning	Self-supervised Learning	↪ (4)	↗ (5)	↔ (3)	12
	Meta-learning	↔ (3)	↔ (3)	↗ (5)	11
	Fine-tuning	↔ (3)	↔ (3)	↘ (2)	8
	Human Value Alignment	△ (1)	△ (1)	△ (1)	3
Emerging Trends	Multimodal Learning	↗ (5)	↔ (3)	↗ (5)	13
	Interactive and Cooperative AI	↪ (4)	↔ (3)	↗ (5)	12
	AGI Development	↪ (4)	↪ (4)	↔ (3)	11
	AGI Containment	△ (1)	△ (1)	↗ (5)	7
	Thinking Models	↪ (4)	↗ (5)	↗ (5)	14
	Agentic AI	↪ (4)	↪ (4)	↗ (5)	13
	Test Time Optimization	↗ (5)	↪ (4)	↗ (5)	14

encapsulates these findings, including revised scores for each subdomain, thereby presenting a holistic map of how generative AI has evolved in recent months.

7.2.1. Impact on Model Architecture

Transformer Models: Transformer models have been scored with a redirection requirement (↪) of 4 in both MoE and AGI contexts and a relevance (↔) of 3 in multimodality, leading to an overall score of 11. These models, forming the backbone of many AI architectures, continue to be effective for handling complex input sequences. Nevertheless, to harness the emergent multistep reasoning (as in thinking models) and modular expansions (e.g., MoE gating layers), transformers require strategic adaptation in design and training methodologies.

Recurrent Neural Networks (RNNs): RNNs face a potential decmidrule in relevance, as indicated by their scores: likely to become redundant (↘) 2 in both MoE and AGI contexts while still relevant (↔) 3 in multimodality, resulting in a total of 7. While RNNs excel at certain types of sequential or stream-based data, their limitations in capturing long-range dependencies and the rise of more efficient transformer-based approaches continue to overshadow RNNs in large-scale tasks. They may retain a niche for highly specialized or constrained scenarios.

MoE: MoE models have scored a consistent relevance (↔) of 3 for their own development and a score of 5 (emerging direction, ↗) in multimodality. In the context of AGI, they require redirection (↪) of 4, summing to 12 overall. Their sparse gating approach is particularly well suited to specialized subtasks and high-parameter efficiency, making them key candidates for advanced LLMs and multimodal deployments. At the same time, integrating MoE models into agentic frameworks demands dynamic load balancing, flexible routing, and possibly test-time compute expansions to adapt to more generalized cognitive tasks.

Multimodal Models: These models received high emerging direction (↗) scores of 5 in both MoE and AGI domains, together with a relevance (↔) of 3 in existing multimodality frameworks, leading to an overall score of 13. The integration of MoE and the push toward AGI create new pathways for multimodal processing, including omni-modal intelligence that spans text, image, audio, and possibly video streams. Recent frontier models like Gemini and GPT-4o highlight how advanced architectures can concurrently handle multiple sensory inputs for more robust reasoning and agentic interactions.

7.2.2. Impact on Training Techniques

Supervised Learning: Supervised learning scored a redirection (↪) of 4, relevance (↔) of 3 in multimodality, and potential redundancy (↘) of 2 in AGI, summing to 9. Although supervised datasets remain essential in many applications, LLMs and agentic systems rely increasingly on self-supervised or reinforcement-driven approaches. Consequently, supervised learning is gradually being supplanted by these more autonomous paradigms in certain advanced or generalizable tasks.

Unsupervised Learning: Unsupervised learning obtains a redirection requirement (↪) of 4 in MoE and AGI and maintains relevance (↔) of 3 in multimodality, totaling 11. New forms of generative pretraining, such as masked-token modeling and contrastive approaches, remain central to large-scale LLMs. The inherently flexible nature of unsupervised learning is especially critical in multimodal expansions, where labeled data are often sparse.

Reinforcement Learning: Reinforcement learning is still relevant (↔) with 3 in MoE, requiring redirection (↪) 4 in multimodality, and emerging (↗) 5 in AGI, yielding 12. Of particular note is the agentic usage of RL for fine-tuning LLMs with environmental feedback (e.g., RLHF or tool usage). In multimodal environments, RL can coordinate multiple streams of perception and action. As AGI-level capabilities mature, RL frameworks are likely to feature even more prominently in how these models learn and act in unstructured tasks.

Transfer Learning: Transfer learning shows relevance (↔) of 3 in MoE, emerging (↗) 5 in multimodality, and redirection (↪) 4 in AGI, totaling 12. Given the explosive scale of LLM pretraining, transfer learning proves indispensable for tailoring these giant networks to downstream tasks or domains. In a future AGI paradigm, it may expand to encompass not merely static fine-tuning but also real-time domain assimilation, bridging specialized knowledge across multiple tasks.

7.2.3. Impact on Application Domains

Natural Language Understanding (NLU): NLU remains relevant (↔) with 3 in MoE and multimodality and emerging (↗) 5 in AGI, resulting in a total of 11. The MoE’s capacity to handle massive amounts of language data enhances NLU’s precision; multimodal setups enable more holistic language understanding, including cross-linguistic or context-imbued tasks. Future AGI progress is poised to radically expand NLU into areas requiring advanced semantics, pragmatics, and reasoning.

Natural Language Generation (NLG): NLG remains relevant (↔) of 3 in MoE, requiring redirection (↪) 4 in multimodality, and emerging (↗) 5 in AGI, summing to 12. NLG stands at the heart of new product categories in content creation and interactive media. Achieving fluid, contextually coherent, and ethically aligned generation demands advanced gating, Chain-of-Thought reasoning, and methodical integration of multimodal signals.

Conversational AI: Conversational AI is assigned redirection (↪) of 4 in MoE and emerging (↗) 5 in both multimodality and AGI, leading to 14. Frontier LLMs such as ChatGPT, Bard, Claude, and Gemini have already showcased how combining a LLMl with tool usage, advanced search, or multimodal inputs can yield more human-like dialogue, including deep contextual understanding and real-time adaptation.

Creative AI: Creative AI scores 4 (redirection) for MoE and 5 (emerging) in both multimodality and AGI (totaling 14). With the integration of multimodal data, generative video, audio, and 3D content expand beyond mere text and image creation. Developments like Google’s Veo or OpenAI’s Sora highlight the growing interest in bridging large language modeling with creative endeavors, potentially leading to novel forms of multimodal artistic generation.

7.2.4. Impact on Compliance and Ethical Considerations

Bias Mitigation: Bias mitigation shows a redirection (↪) of 4 in MoE and multimodality and emerging (↗) 5 in AGI, totaling 13. As systems become more modular (via MoE) and integrate multimodal data, new forms of bias can emerge or amplify. Researchers must embed advanced bias detection and auditing mechanisms early in the development cycle, especially for Q*-like or AGI-like pipelines.

Data Security: Data security remains relevant (↔) 3 across MoE, multimodality, and AGI. Even though distributed or multiexpert architectures amplify potential vulnerabilities, foundational practices like encryption, access control, and robust data governance persist as mainstays, with possible incremental updates needed for emerging large-scale or agentic systems.

AI Ethics: AI ethics is marked redirection (↪) 4 in MoE and multimodality but inherently unresolvable (△) 1 in AGI (score of 9 overall). As soon as LLMs gain agentic capabilities, existential and normative questions escalate—ranging from accountability to moral agency. The complexities of AGI-level systems suggest that fully “resolving” ethics may remain elusive, likely requiring ongoing cross-disciplinary discourse.

Privacy Preservation: Privacy preservation sees a redirection (↪) of 4 across MoE, multimodality, and AGI, reaching 12. The MoE’s distributed nature or agentic AI’s autonomous data usage can complicate the enforcement of data minimization and confidentiality principles. More sophisticated solutions, possibly hardware-accelerated and cryptographically enriched, are needed to maintain user trust and legal compliance.

7.2.5. Impact on Advanced Learning

Self-supervised Learning: For the MoE, self-supervision demands redirection (↪) 4; in multimodality, it is emerging (↗) 5; and it is still relevant (↔) 3 in AGI contexts (totaling 12). While these massive generative systems have scaled primarily through self-supervised paradigms, future developments in multimodal and agentic contexts may require novel objectives that handle dynamic, heterogeneous data and real-time tasks.

Meta-learning: Meta-learning remains relevant (↔) 3 for MoE and multimodality but is emerging (↗) 5 in AGI (score of 11). Adaptive “learning to learn” becomes more critical as systems approach general intelligence. Techniques that allow for the quick assimilation of new tasks, especially in real-world open-ended domains, become a priority research area.

Fine-tuning: Fine-tuning remains still relevant (↔) 3 in MoE and multimodality but is likely to become redundant (↘) 2 in AGI, resulting in a total of 8. While domain-specific fine-tuning (RLHF, specialized data, etc.) is widely used today, advanced LLMs with agentic or test time adaptive abilities may gradually diminish its necessity, particularly for broad tasks requiring real-time self-updating strategies.

Human Value Alignment: Human value alignment remains an inherently unresolvable (△) challenge across MoE, multimodality, and AGI, each scoring 1 (totaling 3). As LLMs expand in scope and autonomy, reconciling cultural, moral, and individual values continues to defy definitive technical solutions. Instead, ongoing participatory governance and multistakeholder collaborations are imperative.

7.2.6. Impact on Emerging Trends

Multimodal Learning: Multimodal learning earns emerging (↗) 5 in MoE and AGI and retains relevance (↔) 3 in existing multimodal frameworks, totaling 13. The surge of large-scale multimodal LLMs reflects the growing realization that intelligence emerges more richly when multiple sensory channels are integrated. Innovations like visual–textual Chain-of-RTought or audio–text coherence are key research thrusts.

Interactive and Cooperative AI: This subdomain is requiring redirection (↪) 4 in MoE and still relevant (↔) 3 in multimodality but emerging (↗) 5 in AGI, reaching 12 overall. Multiagent scenarios, co-creative systems, and user-centric interactions demand new approaches to model interpretability, memory integration, and scenario planning—particularly in bridging agentic LLMs with human collaborators.

AGI Development and Containment: AGI development requires redirection (↪) 4 in MoE and multimodality yet remains at its own frontier (↔) 3 in AGI, scoring 11. Expanding beyond domain-specific tasks, AGI aims for universal reasoning and planning. Meanwhile, AGI containment is not required to be solved (△) in MoE and multimodality contexts but emerging (↗) 5 in AGI (totaling 7). As AI moves closer to agentic capabilities, safe deployment and fail-safe control measures (both technical and policy-based) become significantly more urgent.

Thinking Models and Agentic AI: Thinking models (test-time compute, multistep reasoning, etc.) and agentic AI (tool usage, autonomous planning, etc.) represent newly emphasized trends that heavily reshape the research landscape.

• Thinking Models: Demand re-engineering of existing architectures, such as the MoE, to enable iterative solution-finding during inference. They also push multimodal and AGI work toward expanded context windows, hierarchical reasoning, and memory-augmented transformations.
• Agentic AI: Extends LLM capabilities by enabling them to plan actions, execute external commands, and adapt to real-time feedback. This raises new research questions about controllability, misalignment, and safety, linking to the emergent concept of test time optimization and dynamic resource allocation.

Combined, these two strands accelerate the evolution of generative AI into more interactive, autonomous, and context-resilient systems, further highlighting the necessity for robust ethical and governance frameworks.

Another emerging trend, thinking models, is scored as requiring redirection (↪) in MoE (4) because iterative reasoning steps must integrate effectively with sparse expert layers, while in multimodality and AGI, it is an emerging research direction (↗), with each having a score of 5. This reflects the heightened need for multistep and context-dependent reasoning across diverse sensory data, as well as the pursuit of self-reflective intelligence.

Similarly, agentic AI is identified as requiring redirection (↪, 4) in both MoE and multimodality, highlighting that autonomous goal-driven behavior imposes new design and training considerations for these architectures. For AGI (↗, 5), agentic capabilities represent a key priority, driving research toward adaptive planning and self-directed action.

Lastly, test time optimization is seen as an emerging direction (↗, 5) in both MoE and AGI, indicating that adaptive inference methods—where models dynamically adjust compute based on task complexity—can synergize well with sparse expert systems and more advanced reasoning. In multimodality, it requires redirection (↪, 4), underscoring the necessity to balance on-demand resource allocation across multiple data types.

Overall, the impact analysis underscores a rapidly evolving landscape where older paradigms (e.g., RNN-based sequence modeling, purely supervised training, etc.) are partially supplanted by more flexible, large-scale, and hierarchical approaches. Meanwhile, new frontiers in multistep reasoning, agentic behaviors, and multimodal integration open both opportunities for real-world innovation and urgent ethical/safety considerations. As such, generative AI research appears poised to undergo further seismic shifts, especially as developers and researchers race to adapt existing methods to the complexities of multidomain, multimodal, and multistep problem solving that define the next generation of advanced AI systems.

8. Emergent Research Priorities in Generative AI

Recent breakthroughs in LLMs, multimodal learning, and agentic AI have reshaped the generative AI research landscape, prompting new directions and emphasizing the scalability and specialization of MoE approaches [106,186,324]. While the previous sections highlight the importance of the MoE, reinforcement learning, and multimodality, this section integrates these topics within broader trends in generative AI and AGI.

8.1. Emergent Research Priorities in MoE

(1)

Advanced MoE Architectures for Multimodal Models in Model Architecture:

MoE designs, which distribute computational load across specialized experts, increasingly combine textual, visual, and other data modalities to address tasks with complex data streams [322]. Such integration aims to strengthen the model’s capacity for domain-specific and cross-domain reasoning. Efforts to incorporate agentic frameworks and test-time compute have demonstrated the potential of MoE systems to dynamically route inputs across diverse experts or modalities, which is essential for both specialized and generalized AI scenarios.

(2)

Synergy Between MoE and Multimodal Learning:

Building on progress in large-scale LLMs, MoE in multimodal learning leverages gating mechanisms to selectively activate relevant experts when processing text, image, audio, or video streams [125,323]. This capacity aligns well with the growth of generative models that can synthesize richer content, indicating new directions in combining MoE with interactive and real-time inference for advanced applications (e.g., robotics or immersive AI).

(3)

Investment Trends in MoE:

Funding patterns reflect a shift toward specialized architectures capable of multisensory integration and real-time adaptation. This trend is evident as major tech companies and research organizations invest in large-scale MoE-based systems [325], driving further innovation in training optimization (e.g., load balancing, memory-efficient gating, etc.) and performance enhancements for extremely large models.

8.2. Emergent Research Priorities in Multimodality

Multimodal generative AI entails developing models that effectively fuse text, images, audio, and more. Building on the synergy with MoE, we identify the following key developments:

• MoE-Driven Multimodal Pipelines: MoE models integrated within transformers can help manage complex data distributions by assigning modalities or subtasks to specialized experts [106]. This approach mitigates the computational overhead of large multimodal embeddings while preserving flexibility in learning.
• Transfer Learning for Cross-Modality: Transfer learning remains vital, enabling models to reuse representations from one modality to augment learning in another [87]. Emerging research focuses on bridging these modalities via universal encoders or decoders, which is in line with large LLM expansions into audio, vision, and beyond.
• Conversational and Creative AI Integration: With the advent of advanced generative models (e.g., ChatGPT, Sora, Gemini, etc.), multimodal AI increasingly supports natural interactions involving both text and visuals, further stimulating research on collaborative content generation and deeper user engagement.
• Self-Supervised Paradigms: Self-supervised learning plays a pivotal role in scaling to massive unlabeled multimodal datasets. Future work explores how gating or MoE logic can tailor representations to modality-specific nuances while still maintaining a shared latent space.

These advancements also drive transformations in educational programs, encouraging AI curricula to emphasize cross-disciplinary expertise, particularly around multimodal fusion and large-scale data management.

8.3. Emergent Research Priorities in AGI

The quest for AGI has sparked a surge in interdisciplinary research, focusing on human-like understanding, autonomy, and creative problem solving [67,316]. Below are key areas experiencing renewed attention:

• Reinforcement Learning for Agentic Reasoning: AGI aspirations hinge on adaptable models that learn from and respond to unstructured environments [352]. Combining MoE-based LLMs with reinforcement signals is emerging as a way to achieve both specialized skill (via experts) and generalizable decision making.
• Complex Application Domains: AGI extends beyond standard NLP tasks, targeting advanced medical diagnosis, scientific discovery, and highly creative endeavors. Natural Language Understanding (NLU), generative text, and multisensor integration underscore the potential for bridging multiple fields—while simultaneously raising new ethical and safety concerns [28].
• Bias Mitigation and Ethical Frameworks: As systems scale, so do risks of amplified biases, security loopholes, and accountability gaps. Aligning AI with fairness and transparency remains a priority, requiring cross-disciplinary collaboration [340].
• Meta-Learning and Emergent Trends: AGI solutions increasingly explore meta-learning, aiming to equip models with the capacity to rapidly adapt to new tasks or contexts [338]. Equally crucial is the integration of “AGI containment” or safety strategies, reflecting the need to manage advanced autonomous models responsibly.

Along with these trends, funding trajectories reveal a growing interest in AGI’s foundational technologies, such as MoE-based inference pipelines, large-scale reinforcement learning environments, and advanced domain adaptation. By directing investment toward robust, transparent, and ethically sound research, the field moves closer to harnessing generative AI’s broader societal impact.

9. Practical Implications and Limitations of Generative AI Technologies

Recent advancements in generative AI—such as ultra-large Mixture-of-Experts (MoE) models, multimodal architectures, and increasingly agentic AI frameworks—underscore both the vast potential and the operational constraints of these technologies. This section expands on the computational challenges inherent in training and deploying advanced AI models while detailing real-world applications, market readiness, and key limitations.

9.1. Computational Complexity and Real-World Applications of Generative AI Technologies

9.1.1. Computational Complexity

Generative AI systems, encompassing MoE models, multimodality, and prospective AGI paradigms, present unique computational challenges in terms of scaling, memory requirements, and inference throughput. Pertaining challenges are listed below:

• Processing Power Requirements: Advanced generative AI models, including MoE architectures and large-scale LLMs with multistep reasoning, typically demand substantial compute resources [353]. The need for efficient GPU/TPU provisioning grows more acute in scenarios involving multimodal data or test time scaling, potentially leading to significant infrastructure costs.
• Memory Usage in AI Modeling: A critical challenge in training and deploying large-scale AI models, particularly those with sparsely activated experts or multisensory inputs, lies in substantial GPU and VRAM requirements. Unlike main system memory, VRAM remains relatively constrained, posing a bottleneck for massive multimodal models or large MoE solutions. Novel methods that offload expert parameters or use dynamic expert loading can mitigate these challenges.
• Scalability and Efficiency: Addressing scalability in generative AI, especially in MoE-based and AGI contexts, involves optimizing load management and parallel processing [322]. Research on specialized routing, advanced pipeline parallelism, and memory-optimizing strategies is critical for real-world deployments in healthcare, finance, and education.

9.1.2. Real-World Application Examples of Generative AI Technologies

Despite the computational cost, generative AI’s potential continues to reshape diverse sectors, often outpacing conventional analytics or rule-based systems. The following lists some applications:

• Healthcare: Generative AI powers diagnostic imaging, personalized treatment recommendations, and disease forecasting [354]. However, reliance on large training corpora raises privacy questions and can demand specialized hardware for medical-grade model inference.
• Finance: AI-driven fraud detection and algorithmic trading illustrate high-impact, data-intensive applications [355]. At the same time, decision-making transparency and robust data governance become essential to counter possible biases or hidden vulnerabilities in generative pipelines.
• Education: Generative AI fosters adaptive learning, automated content creation, and personalized feedback loops. While these systems promise scalable tutoring and resource-saving benefits, ethical debates around academic integrity persist. Concerns arise regarding the potential of AI-generated content (AIGC) to displace educators or compromise academic authenticity.

9.2. Commercial Viability and Industry Solutions

9.2.1. Market Readiness

Market adoption of generative AI depends on factors, including cost, domain alignment, and workforce readiness, to integrate advanced ML pipelines into existing workflows.

• Cost Analysis: Running large generative models—especially MoE or multimodal frameworks—can strain budgets due to hardware needs and specialized engineering talent.
• Accessibility and Deployment: Enterprise readiness varies. Some industries (e.g., tech, finance, etc.) rapidly embrace AI, while others face skill gaps and infrastructural barriers.
• User Adoption Trends: Public-facing generative AI tools, such as ChatGPT, show swift user adoption. However, domain-specific solutions require specialized training data, expansions in interpretability, and robust MLOps pipelines to gain user trust.

9.2.2. Existing Industry Solutions

Generative AI reshapes sectors by offering unprecedented automation and content generation:

• Sector-Wise Deployment: From content creation (marketing, design, etc.) to code generation and robotic process automation, generative AI demands re-evaluation of creative ownership and IP rights.
• Impact on Market Dynamics: Traditional players face competition from AI-native startups, while new business models, such as AI-as-a-Service (AIaaS), form around generative capabilities.
• Challenges and Constraints: Fundamental issues like scalability, data management complexity, and privacy concerns persist, underscoring the importance of robust governance, standard-setting bodies, and multistakeholder dialogues.

9.3. Limitations and Future Directions in Generative AI

9.3.1. Technical Limitations

As generative AI expands into real-time, multimodal, and agentic domains, several technical gaps become increasingly salient:

• Contextual Understanding: Despite LLM breakthroughs, common sense reasoning and long-horizon context modeling remain weak points, spurring research into memory-augmented architectures.
• Handling Ambiguous Data: Real-world data often contain inconsistencies or missing attributes. Reliability in these conditions may demand robust inference techniques, e.g., Bayesian or reinforcement-based training, to handle uncertainty.
• Navigating Human Judgment: Even though generative AI can interpret policies and procedures, it falls short in reproducing nuanced human values or in assessing complex legal/political implications. Biased or manipulative usage of AI-generated content (AIGC) can lead to skewed decision making.

9.3.2. Future Research Directions

Addressing these shortcomings—while harnessing generative AI’s strengths—depends on continued multidisciplinary innovation:

• Improved Contextual Understanding: Research on Chain-of-Thought prompting, hierarchical gating, and multihop inference can deepen AI’s ability to interpret extended or complex queries.
• Robust Handling of Ambiguity: Model architectures that incorporate reliability estimates or adversarial training may yield more stable performance under ambiguous data conditions.
• Ethical Integration of AIGC in Legal and Political Arenas: As generative models permeate high-stakes environments, developing frameworks that bolster transparency, mitigate bias, and validate correctness is imperative [356]. Researchers must also consider socio-technical complexities—such as corruption or manipulative usage—where advanced generative models could inadvertently amplify systemic inequities.

In conclusion, generative AI’s commercial viability and real-world impact hinge on strategic investment in computational efficiency, regulatory guidance, and ethical oversight. Although future models may significantly augment or automate tasks in healthcare, finance, and education, they also challenge existing norms around accountability, interpretability, and trust. Addressing these technological and societal questions will define the next phase of AI’s adoption and ensure that generative systems serve as a beneficial rather than disruptive force.

10. Impact of Generative AI on Preprints Across Disciplines

The widespread success and rapid commercial availability of generative AI models—exemplified by ChatGPT—have accelerated the production and dissemination of academic manuscripts to unprecedented levels. This phenomenon is particularly evident in the surge of preprints on platforms like arXiv (Figure 7), where submissions in the cs.AI category demonstrate a steep climb [357,358]. As generative AI becomes integral to both research and writing processes, the traditional mechanisms of peer review struggle to keep pace, fueling concerns about overall scientific rigor and the potential propagation of unverified or substandard results.

A notable consequence is the bottleneck now observed in scholarly communication: the peer-review workflow, historically a gatekeeper for quality assurance, lacks the bandwidth to manage the deluge of rapidly generated manuscripts. Parallelly, generative tools (e.g., ChatGPT, Claude, etc.) expedite the manuscript drafting process, sometimes obfuscating author contributions or overshadowing methodological weaknesses [357,358]. This proliferation of AI-facilitated or AI-authored manuscripts—spanning not only computer science but also disciplines like biology, social sciences, and engineering—poses a critical test for the resilience and adaptability of academia.

Furthermore, the swift release of preprints often outpaces the capacity for thorough validation or replication, complicating evidence synthesis and systematic reviews [359,360]. High-volume preprint posting can saturate scholarly communities, particularly in emerging fields such as multimodal AI, Mixture-of-Experts (MoE), and even the nascent exploration of AGI. Consequently, this exponential growth can diffuse focus, rendering it challenging for experts to track crucial developments or filter out lower-quality submissions. More problematic is that many preprints lack a robust editorial or retraction infrastructure, potentially perpetuating flawed methods and biases.

Peer Review at a Crossroads: The accelerated publishing cycle has magnified the limitations of traditional peer review, which is a bottleneck that is underscored by limited reviewer availability, extended turnaround times, and the specialized nature of modern AI research [361]. In areas such as MoE-based language modeling or agentic AI, only a handful of qualified reviewers may be available, resulting in possible conflicts of interest or undue delays. Consequently, the academic community grapples with the risk of unverified findings circulating widely and influentially.

Hybrid Models of Validation: Amid these challenges, novel approaches to vetting emerging AI research are under active exploration. One proposal envisions hybrid models of review that combine preliminary, community-driven assessments—akin to product review or code repository feedback—with more formal peer review [362]. In this structure (Figure 8), preprints receive rapid user commentary, highlighting potential strengths or flaws early in the dissemination process. Editors and formal reviewers subsequently refine these critiques, integrating peer evaluations focused on methodological rigor, ethical considerations, and replicability. Advanced AI tools might also be enlisted to flag textual inconsistencies or duplication, which is a measure that could streamline triage and mitigate unethical practices like ghostwriting or auto-generation of entire sections.

While no single strategy can fully address the scale and complexity of today’s AI publishing surge, these emerging paradigms offer a blueprint for adapting academic norms. The shift to more open, community-oriented, and continuous review processes could provide a partial remedy to the “publish-first, validate-later” culture enabled by generative AI. Equally important is the establishment of best practices, such as labeling AI-generated text, clarifying author roles, and adopting consistent ethics statements. Taken together, these innovations may help sustain academic rigor amid the accelerating influx of AI-driven research.

11. Conclusions

This roadmap survey has embarked on an exploration of the transformative trends in generative AI research, particularly inspired by recent advancements in producing Large Language Models that can solve increasingly complex reasoning tasks via a scaling mechanism that can flexibly apportion more compute resources at inference time. Our analysis highlights that such progress has led to a paradigm shift, which is complemented by additional technological innovations such as Mixture of Experts, agentic AI, multimodal and adaptable AI systems, and the pursuit of AGI. These advancements signal a future where AI systems could significantly extend their capabilities in reasoning, contextual understanding, and creative problem solving. This study reflects on AI’s dual potential to either contribute to or impede global equity and justice. The equitable distribution of AI benefits and its role in decision-making processes raise crucial questions about fairness and inclusivity. It is imperative to thoughtfully integrate AI into societal structures to enhance justice and reduce disparities. Despite these advancements, several open questions and research gaps remain. These include ensuring the ethical alignment of advanced AI systems with human values and societal norms, which is a challenge compounded by their increasing autonomy. The safety and robustness of AGI systems in diverse environments also remain as significant research gaps to explore. Addressing these challenges requires a multidisciplinary approach, incorporating ethical, social, and philosophical perspectives.

Our survey has highlighted key areas for future interdisciplinary research in AI, emphasizing the integration of ethical, sociological, and technical perspectives. This approach will foster collaborative research, bridging the gap between technological advancement and societal needs and ensuring that AI development is aligned with human values and global welfare. The roles of recent technological breakthroughs in generative AI have been identified as significant, as their advancements can enhance model performance and versatility, as well as pave the way for future research in areas like ethical AI alignment and AGI. As we forge ahead, the balance between AI advancements and human creativity is not just a goal but a necessity, ensuring AI’s role as a complementary force that amplifies our capacity to innovate and solve complex challenges. Our responsibility is to guide these advancements toward enriching the human experience, aligning technological progress with ethical standards and societal well-being.