Defending Against AI Threats with a User-Centric Trustworthiness Assessment Framework

Abstract

This study critically examines the trustworthiness of widely used AI applications, focusing on their integration into daily life, often without users fully understanding the risks or how these threats might affect them. As AI apps become more accessible, users tend to trust them due to their convenience and usability, frequently overlooking critical issues such as security, privacy, and ethics. To address this gap, we introduce a user-centric framework that enables individuals to assess the trustworthiness of AI applications based on their own experiences and perceptions. The framework evaluates several dimensions—transparency, security, privacy, ethics, and compliance—while also aiming to raise awareness and bring the topic of AI trustworthiness into public dialogue. By analyzing AI threats, real-world incidents, and strategies for mitigating the risks posed by AI apps, this study contributes to the ongoing discussions on AI safety and trust.

1. Introduction

In today’s rapidly evolving landscape, Artificial Intelligence (AI) applications are pervasive, profoundly impacting our daily lives [1]. From the personalized interactions of chatbots [2] to the captivating yet potentially deceptive world of deepfakes [3], AI has become a leading force. The democratization of AI, a process that makes cutting-edge technologies widely accessible, is transforming the way we work, communicate, and create [4].

However, this rapid integration of AI also brings significant concerns about the trustworthiness of these systems, particularly regarding security, privacy, and ethics, manifested in various dimensions [5]. Security concerns involve the susceptibility of AI applications to exploitation by malicious actors, potentially resulting in unauthorized access and data breaches. Privacy-related issues are centered on challenges associated with the collection, storage, and utilization of personal data by AI systems, often lacking transparent user consent, risking unauthorized surveillance or the exposure of sensitive information [6]. Ethical concerns include challenges arising from biased algorithms, discriminatory practices, or unintended consequences in AI decision-making, with the potential to adversely impact individuals and society [7].

The widespread popularity of AI applications is often driven by their positive user experience. Although these applications express approval of their ease of use and engaging interfaces, this positive experience may obscure critical risks related to trustworthiness. Users might trust these apps based entirely on their user-friendly design [8], potentially being unaware of significant underlying issues.

Additionally, current methods to control and assess the trustworthiness of AI largely rely on technological countermeasures [9], regulatory frameworks [10], and trustworthiness assessment tools [11]. Technological measures include advanced algorithms for data security and privacy [12]. Regulatory frameworks, such as the General Data Protection Regulation (GDPR) and other initiatives [13], set legal standards for AI operations. Moreover, trustworthiness assessment tools, such as the Assessment List for Trustworthy Artificial Intelligence (ALTAI) [14], offer guidelines to businesses enabling them to evaluate their AI systems across multiple dimensions.

While such approaches to evaluating and safeguarding AI systems offer significant benefits, they are primarily designed for and addressed to experts [14,15,16,17]. However, as AI applications become more popular and integral to daily life, users are increasingly exposed to potential risks and ethical concerns associated with these technologies. Unfortunately, many users remain unaware of these risks and lack the knowledge to protect themselves effectively [18]. Moreover, while there have been efforts to measure AI trustworthiness through measuring user experience [8], there are still limited practical tools available that allow users to directly assess AI trustworthiness based on their own perceptions. This gap highlights a need for a more intuitive and user-centric approach, one that raises awareness about AI trustworthiness, while equipping users with straightforward methods to evaluate and navigate AI technologies confidently.

Towards narrowing this gap, this paper introduces the AI User Self-Assessment Framework, designed to bridge the divide between complex, expert-driven evaluations and the practical, everyday experiences of users with AI technologies. The proposed framework empowers users by offering a straightforward and practical tool to assess the trustworthiness of the AI applications they encounter regularly. By distilling complex trustworthiness dimensions into accessible criteria, the framework enhances user awareness of potential risks and encourages proactive, informed engagement.

As AI technologies become increasingly embedded in daily life, we envision this framework being integrated into rating systems and community-driven platforms. Such integration would allow users to contribute to and benefit from collective evaluations of AI apps, ensuring that trustworthiness assessments are aligned with broader societal values and user expectations.

The remainder of this article is organized as follows: Section 2 presents a literature review on the threats posed by popular AI applications, showcasing real-world instances where AI trustworthiness has been compromised. Additionally, this section discusses the relationship between AI trustworthiness and user experience. While users generally report positive experiences with popular AI applications, limited direct measurements exist regarding their trustworthiness. Section 3 explores strategies for mitigating risks and ensuring AI trustworthiness, noting that existing controls often target expert users and leave a gap for average users. Next, Section 4 introduces our proposed AI User Self-Assessment Framework, a practical tool designed to help average users evaluate the trustworthiness of AI applications based on their personal experiences. This section details the methodology we followed to develop and implement the framework. Finally, Section 5, draws conclusions and discusses potential directions for future improvements.

2. Threats and Trustworthiness of AI Applications

This section explores the complex landscape of threats and trustworthiness in AI applications. We start by examining the various risks associated with these technologies through theoretical discussions and real-world examples. While our focus includes generative AI apps, known for their data-intensive nature and widespread attention, we also address broader issues relevant to AI applications more generally. Given that many AI apps become popular due to their positive user experience and usability, we subsequently analyze how this user satisfaction may obscure underlying concerns related to different dimensions of AI trustworthiness.

2.1. Generative AI Threats

Exploring AI democratization, we encounter numerous threats linked to the popularity of AI apps. The rise in the use of generative AI chatbots, exemplified by applications such as ChatGPT, has attracted malicious actors who exploit this trend to spread malware [19]. A study conducted by Meta identified 10 different types of malware utilizing ChatGPT to target and compromise users’ devices since March 2023 [20]. Additionally, AI-driven phishing attacks have evolved into powerful tools that can manipulate users into revealing sensitive information or taking harmful actions. AI language models, such as ChatGPT or Bing’s GPT-4-powered Chat, rely heavily on the prompts they receive. While some prompts are crafted to enhance the model’s performance across a range of tasks, others can hide intentions that pose real, tangible threats [21]. For instance, specific prompts can transform AI chatbots into tools for creating convincing phishing websites that imitate famous brands, evading advanced anti-phishing systems [22]. Another example includes the utilization of AI chatbots to generate deceptive emails or messages that convincingly mimic trusted sources, thus increasing the risk of users falling victim to phishing attempts. The ability of chatbots to remember context in conversations inserts an additional layer of concern, enabling the creation of more convincing phishing emails from a simple prompt [23].

Moreover, the biases inherent in AI systems pose a considerable threat to users and consequently to society as a whole. Researches highlight that while LLMs can outperform humans in tasks with explicit mathematical or probabilistic elements, their responses reveal perpetuation of numerous biases when confronted with intricate, ambiguous, and implicit problems [24]. Beyond language models, the world of Text-to-Image (T2I) systems also raise concerns about their impact on society. Luccioni et al. [25] have extensively studied this issue, revealing a notable overrepresentation of whiteness and masculinity in images created by popular T2I systems (DALL·E 2, Stable Diffusion v 1.4 and v 2). Drawing attention to the broader implications, it becomes alarming when AI applications are adopted at young ages, as bias is ingrained and conveyed, shaping an audience oblivious to the existing threats posed by such technologies. This lack of awareness can result in biases being accepted without scrutiny, thereby intensifying the societal impact of AI systems.

Generative AI models can additionally be exploited for the generation of harmful content, posing a significant risk to individuals’ reputations and societal trust. LLMs are particularly susceptible to data poisoning, which allows for the injection of hateful speech targeting specific individuals or entities. Since such models operate based on statistical patterns rather than possessing a true understanding of the world, they can be extremely susceptible to generating false information, as well as perpetuating biased and hateful statements [26]. This flaw can blur the distinction between genuine and false content, further highlighting how AI-generated text smoothly merges into our online environment. A recent study by Qu et al. [27] conducts a safety assessment of T2I models, focusing on the generation of unsafe images and hateful memes. The findings reveal that T2I models can produce substantial rates of unsafe images, even with seemingly harmless prompts. This discovery emphasizes the concerning capacity of these models to generate hateful content that closely resemble real-world instances, thus elevating the risk of the dissemination of hate campaigns.

In the same context, the phenomenon of “hallucination” within LLMs carries significant implications, involving the generation of text that is inaccurate, nonsensical, or entirely fictional [28]. Unlike databases or search engines, LLMs do not cite sources for their responses; instead, they derive text through extrapolation from the provided prompt. This extrapolation, although not necessarily backed by training data, represents the most correlated output based on the given prompt. The severity of this issue lies in the potential for LLMs to produce misleading or entirely fabricated information. For example, an LLM prompted with a historical events question, generates a response inaccurately describing crucial details. Users, unaware of the model’s limitations, may accept the information as accurate, perpetuating the dissemination of false or misleading content.

Similar concerns apply considering the fact that AI apps are being employed to propagate misinformation and manipulate public opinion in the context of critical matters, such as health or climate change. Malicious actors exploit the capabilities of AI models, especially generative neural networks, to generate persuasive and convincing content that disseminates false information or manipulates people’s beliefs. Henley et al. reveal the growing use of LLMs in generating articles on news websites. Synthetic articles produced by these models are found to be increasingly common, while social media users are increasingly engaging with these synthetic articles [29].

In educational settings, AI’s potential benefits, particularly in creative areas, coexist with limitations, raising questions about bias, power dynamics, and ethical concerns [30]. Consider, for example, AI-driven recommendation systems in educational software. Although these systems aim to enhance learning experiences by providing personalized content suggestions, there is a risk of reinforcing existing biases or restricting students’ exposure to diverse perspectives. Additionally, the misuse of AI, as highlighted in computer security-oriented specializations, poses a threat to academic integrity, necessitating a balance between leveraging AI as an educational support mechanism and preserving critical thinking in learning environments [31].

2.2. Real-World Examples

Moving beyond the conceptual landscape of AI threats, our study continues with real-world examples, where these conceptual concerns materialize into tangible challenges. By examining notable failures and incidents, our objective is to derive practical insights that underscore the real impact of the discussed threats.

For example, IBM’s Watson Health, despite the ambitious claims about revolutionizing healthcare, resulted in numerous misdiagnoses and treatment options that were not feasible [32]. Its reliance on vague and erroneous data could have resulted in widespread misdiagnoses and ineffective treatment options if it had been widely adopted. Similarly, Amazon’s biased AI resume screening tool exhibited gender bias, favoring male candidates based on keywords found in predominantly male resumes [33]. Had it not been documented and highlighted as failure, the tool could perpetuate gender discrimination and limit opportunities for qualified women if used extensively across various industries. Microsoft’s chatbot Tay, which became racist and sexist due to biased training data and interactions with users [34], could have spread offensive and harmful content if accessible to a larger audience, further amplifying prejudices.

Apart from these widely discussed and criticized examples, cutting-edge generative AI models such as ChatGPT, Stable Diffusion, and DALL·E, currently more accessible to a wider user base, have demonstrated the potential to fail by producing harmful content when deployed without sufficient controls, as evidenced in Figure 1 and Figure 2. These generative AI models are being exploited in jailbreak attacks, where attackers leverage vulnerabilities to surpass the models’ safeguards. In instances where these defenses are breached, harmful behaviors are triggered, posing substantial risks, including the potential compromise of personal data, financial information, and access to sensitive systems. Such AI models can often generate biased content, mirroring societal biases and potentially perpetuating stereotypes that undermine efforts to cultivate inclusivity and diversity.

Reports have highlighted instances where users can manipulate generative AI models, such as chatbots, to generate inappropriate or harmful content, such as crafting a negative review for a business. This manipulation, especially if repeated on purpose, poses a significant threat as it can seriously damage a business’s reputation, underscoring the necessity for robust content filtering mechanisms. Furthermore, the use of AI-generated text in phishing attempts has showcased its capability to craft convincing phishing emails that, when deployed, can readily deceive users. If this technique becomes widely adopted, it has the potential to heighten the efficacy of phishing attacks, significantly elevating the risk of cybersecurity threats for both individuals and organizations.

The potential harm and societal consequences of uncontrolled AI, as discussed by these instances and exemplified in Figure 1 and Figure 2, necessitate robust technological controls driven by adherence to ethical guidelines. Responsible development and use of AI requires a commitment to principles prioritizing security and privacy, ethics, and trustworthiness, aiming for user safety and societal well-being.

2.3. Trustworthiness and User Experience of AI Applications

The trustworthiness of AI applications has collected significant attention as AI systems are increasingly integrated in various sectors. At its core, Trustworthy AI is a framework aimed at ensuring that AI systems are worthy of being trusted based on verifiable evidence concerning their stated requirements. This framework can ensure that users’ and stakeholders’ expectations are met in a transparent, reliable, and ethical manner [35]. In addition to the framework-based view, trust in AI systems can be understood as the belief from the stakeholder’s side in the reliability and credibility of AI agents’ recommendations and responses [36].

According to Shin et al., trust in AI is shaped by users’ perceptions of an AI system’s explainability and causability, which help users understand how decisions are made. When AI systems provide transparent, understandable explanations and consistent, accurate results, users’ trust is strengthened. However, when outputs are biased or lack transparency, trust can be significantly undermined [36].

Moreover, Cesare et al. highlight the strong connection between trust and usability, emphasizing that the system’s interface plays a crucial role in shaping how trustworthy users perceive an application or website to be. In the context of e-commerce, the quality of the interface becomes particularly significant for building trust and trustworthiness is often measured through usability. A clear, easy-to-navigate, and intuitive interface fosters user confidence and security, making users more likely to trust the processes behind the system. This strong link between user experience and trust is essential for promoting long-term engagement and repeat usage of e-commerce platforms [8].

However, these findings suggest that a highly usable interface might encourage users to return, even if hidden vulnerabilities exist beneath the surface, potentially masking risks that go unnoticed. As highlighted in the previous sections, the growing presence of AI in real-world applications has revealed numerous threats and incidents where trustworthiness is being compromised in several of its dimension including security, robustness and privacy [37]. Despite these trustworthiness challenges, AI systems often deliver high usability and positive user experiences, where the perceived trustworthiness of an AI app may be decided based on usability. Many AI applications prioritize user satisfaction through intuitive interfaces and responsive design, yet this usability can mask underlying issues related to trustworthiness. As AI systems become more embedded in daily life, users may unknowingly place greater trust in these applications, equating ease of use with safety and reliability [8]. This disconnect between user experience and the actual trustworthiness of AI highlights the need for more visible, user-friendly mechanisms that allow non-expert users to evaluate the trustworthiness of the AI systems they interact with.

3. Controlling the Risks of AI Apps

In this section, we critically examine the mechanisms designed to control the risks associated with AI applications, focusing on technological controls, regulatory frameworks, and frameworks for trustworthiness assessment. By exploring these areas, as summarized in

Table 1. Summary of control mechanisms for AI applications.

Control Level	Elements	References
Technological	Advanced methods and tools aimed at enhancing AI security, robustness, privacy, and interpretability, including vulnerability detection, response and recovery, and XAI for clearer decision-making.	[15,38,39,40,41,42,43,44,45,46,47]
Regulatory	Laws and guidelines established to govern the ethical use of AI technologies, ensuring compliance with standards for data privacy, transparency, and accountability.	[48,49,50,51,52,53,54]
Trustworthiness	Frameworks and guidelines designed to assess and ensure the ethical principles, transparency, fairness, and overall reliability of AI systems, guiding their responsible development and deployment.	[14,35,37,55,56,57,58]

, we aim to analyze current strategies for mitigating AI-related threats and ensuring the safe, ethical, and responsible use of AI technologies while also highlighting potential limitations and directions for improvement.

3.1. Technological Controls

In response to the evolving threats in the AI landscape, ongoing efforts are dedicated to implementing innovative defensive mechanisms within or with the use of AI models. For instance, recent research suggests a creative approach wherein companies leverage ChatGPT to simulate phishing campaigns, using the convincing results to train their personnel in recognizing real-world phishing attempts [38]. Methods such as Self-Reminder can reduce the success rate of jailbreak attacks by reminding the LLM to respond appropriately [39]. Tools such as BiasAsker have been developed to identify and measure social bias in conversational AI systems [40], while AI-generated content detectors are being introduced to effectively identify fake content and distinguish it from genuine [41,42].

Platforms such as RobustBench offer leaderboards that rank models based on their performance against specific types of threats. Such benchmarking services subject the AI models to specific attacks aiming to provide a standardized way to extensively evaluate the robustness of various AI models, helping researchers and developers assess the effectiveness of their defensive strategies [15].

Moreover, the aspect of data privacy within control mechanisms is explored with innovative methods forcing the AI models to forget, as outlined in [43]. Implementing machine unlearning involves intentionally eliminating or modifying previously acquired patterns or data from AI models. By enabling the system to forget sensitive or malicious information, this strategy reduces the risk of security and privacy concerns.

Additionally, there is an ongoing exploration into enhancing transparency in AI decision-making, combining Explainable AI (XAI) [44,45] alongside Reinforcement Learning with Human Feedback (RLHF) [46]. XAI can provide users with clear insights into the AI decision-making process, while RLHF involves users providing feedback to the AI system, reinforcing positive behavior and correcting undesirable actions.

The concept of Controllable AI, as discussed in recent research, further explores how to handle the explainability and transparency challenges associated with modern AI systems [47]. It proposes practical methods for achieving control over AI systems without necessarily achieving full transparency. This approach emphasizes detecting and managing control loss in AI systems, acknowledging that while complete transparency may be impractical, effective control mechanisms are essential for ensuring the system’s reliability and security.

3.2. Regulatory Controls

A growing emphasis on regulatory frameworks highlights the pressing need for ethical guidelines to govern AI applications, ensuring responsible usage [48,49,50]. As nations formulate guidelines and enact laws to supervise the implementation of AI technologies, these measures emerge as potent tools in shaping the strategies of big tech companies. Regulatory frameworks compel these corporations to adhere to rigorous standards, especially in domains such as data privacy, algorithmic transparency, and ethical considerations. Notably, the European Union’s AI Act incorporates provisions tailored specifically to the use of AI, with a focus on high-risk and prohibited AI applications [51,52]. This regulatory pressure pushes for change, encouraging big tech companies to adopt more respectful, accountable, and ethical practices in their AI endeavors.

In addition to external regulations, AI auditing frameworks can ensure compliance with these standards, as they provide structured methodologies for organizations to assess and manage AI systems’ adherence to regulatory and ethical requirements [53]. Such frameworks typically focus on areas such as privacy protection, data integrity, algorithmic transparency, and ethical considerations and contribute to regulatory controls by enabling organizations to systematically review and address risks associated with AI applications. This type of control can enable organizations to comply with regulatory requirements but also enhance their ability to manage and govern AI technologies responsibly. Thus, while regulatory frameworks set the external standards for AI usage, auditing frameworks complement these efforts by ensuring that internal practices are effectively aligned with these standards.

Following this line of research, Meszaros et al. [54] investigate the legal challenges of generative AI models, and examine how well ChatGPT complies with GDPR in processing personal data. They highlight the importance of the GDPR’s legal bases to ensure compliance. While OpenAI has made significant strides toward meeting GDPR standards, ongoing efforts are essential to fully meet regulatory expectations and safeguard data subjects’ rights. As AI and data protection evolve, a proactive approach to balancing innovation with privacy and compliance is key to developing trustworthy generative AI systems.

3.3. Trustworthiness Assessment

Ensuring the trustworthiness of AI applications is a critical concern, especially as these systems become more ingrained in everyday activities. Several frameworks have been developed to guide experts in evaluating and enhancing the trustworthiness and reliability of AI systems. These frameworks aim to address key ethical principles such as transparency, accountability, and fairness, as well as safety and robustness.

One notable framework is the ALTAI list for trustworthiness assessment [14,55], developed by the European Commission. This framework provides a structured checklist to evaluate AI systems across several crucial dimensions. It emphasizes governance and accountability, requiring clear processes and responsibility definitions. Transparency, as discussed in ALTAI, ensures that AI systems operate in a manner that is clear and understandable, with decisions that are easily traceable by stakeholders. The framework also addresses ethical considerations such as fairness, urging the integration of non-discrimination principles into AI design. Additionally, it highlights the importance of safety, robustness, and privacy protection, advocating for rigorous testing and responsible data handling. Finally, it considers the societal and environmental impacts of AI, encouraging assessments of long-term effects and the adoption of practices to mitigate negative outcomes.

According to a recent review [56], ALTAI represents a notable advancement in translating AI governance principles into actionable assessments, bridging the gap between high-level ethical guidelines and practical implementation. The review highlights ALTAI’s strengths, including its detailed focus on operational governance and its ability to translate ethical principles into measurable standards. However, it has also been criticized for not accommodating organizational maturity differences and lacking integrations with risk assessments. Recommendations for improvement include incorporating peer comparisons and risk-based evaluations to better accommodate varying organizational maturity levels and improve its practical application across different sectors.

Another methodology for trustworthiness assessment, the Z-Inspection process [57], offers a holistic and dynamic approach to evaluating the trustworthiness of AI systems at various stages of their lifecycle. It emphasizes participatory and interdisciplinary assessment methods, incorporating socio-technical scenarios to address ethical issues and ensure alignment with trustworthiness principles. However, the Z-Inspection process is not without its limitations. For instance, its flexibility can lead to inconsistencies in assessments, and the resource-intensive nature may not be feasible for all organizations. Additionally, the process’s reliance on specialized expertise and its dynamic, adaptable approach can pose challenges in terms of scalability and integration with existing regulatory frameworks [58]. These limitations underscore the need for ongoing refinement and adaptation of assessment methodologies to address diverse contexts and emerging challenges in AI trustworthiness.

While frameworks such ALTAI offer valuable guidelines for experts, there are additional dimensions of trustworthiness that need to be addressed to fully align AI systems with user expectations. According to recent research [35], human involvement throughout the AI lifecycle can assist in setting performance limits, correcting errors, and enhancing system reliability. In addition, effective evaluation mechanisms can be crucial in matching user expectations with the system’s actual capabilities, ensuring that users have a clear and accurate understanding of the system’s performance and limitations. Bridging this gap is important to building trust and facilitating the broader acceptance of AI technologies.

Furthermore, a recent study highlights the need for a shift from a performance-driven to a trust-driven model of AI development [37]. This approach aims to address the limitations and trade-offs between different aspects of trustworthiness, such as explainability, robustness, and fairness. It also emphasizes the importance of ongoing research and interdisciplinary collaboration to better understand and address these challenges, ensuring that AI systems can be trusted by all stakeholders involved.

3.4. Limitations and the Need for User Involvement

Although the discussed approaches for controlling AI applications can offer a certain level of protection against AI-related threats, they present several practical limitations.

• echnological defenses often become brittle over time. Malicious actors consistently demonstrate a remarkable ability to stay ahead of defensive mechanisms, continually developing new threats and undermining AI models. This adaptability presents a persistent challenge, continuously testing the effectiveness of existing safeguards and necessitating ongoing improvements to defensive measures.
• Regulations struggle to keep pace with the rapid advancement of technology. This regulatory lag can create gaps in protection, allowing emerging threats to exploit these delays. Moreover, the reactive nature of regulatory updates may not adequately address the speed and sophistication of new AI threats, further intensifying the risk of insufficient safeguards and oversight.
• The technical nature of trustworthiness assessment frameworks limits their accessibility to the general public. Consequently, average users often remain unaware of such frameworks and lack specialized tools to independently evaluate the AI applications they use as of their trustworthiness. This disconnect extends to the specialized terminology used within such frameworks, making it even harder for users to start familiarizing themselves with important aspects such as transparency and privacy that they should expect from AI applications.

With respect to the evolving AI landscape, relying exclusively on technological, regulatory, and formalized trustworthiness assessment controls cannot provide a complete defense against the threats posed by AI. These measures, although useful and crucial, present limitations and can be complex, potentially giving users a false sense of security. To effectively address the dynamic challenges of the AI landscape, a more integrated approach is essential—one that combines existing measures with active user involvement through increased awareness.

Users who interact with AI apps daily are in a unique position to contribute to trustworthiness assessments. Their direct experiences and observations can provide valuable insights that established assessments might miss. Involving users in evaluating and reflecting on the trustworthiness of AI apps they use can help ensure that safety measures are better aligned with real-world interactions, thereby enhancing the effectiveness of AI governance.

4. AI User Self-Assessment Framework: Research Design and Methodology

Given the limitations of existing methods for managing and controlling AI technologies, we introduce in this section the AI User Self-Assessment Framework, a tool designed to place user empowerment at the core of AI trustworthiness evaluation. The proposed framework adopts a user-centric approach, offering a structured set of questions that enable individuals to assess AI applications based on their personal experiences. By focusing on essential dimensions of AI trustworthiness—such as user control, transparency, security, privacy, ethics, and compliance—the framework supports users in evaluating these aspects while promoting deeper understanding of how AI systems operate and impact them.

The following sections detail the specific methodology used in the development of the framework, including the theoretical foundations, design considerations, and the steps that guided its development. Our approach, rooted in a user-centric perspective, aims to provide individuals with a tool to evaluate the trustworhiness of AI applications based on their own experiences and perceptions.

4.1. Conceptual Framework

The development of the AI User Self-Assessment Framework was guided by an extensive literature review that examined the threats associated with popular AI applications, AI trustworthiness, and existing controls to mitigate these risks. From this review, we focused particularly on controls related to AI trustworthiness assessment. This analysis, which included a detailed examination of frameworks such as the ALTAI framework, provided the theoretical foundations by identifying and defining the key dimensions of trustworthiness that were crucial for shaping the framework.

In detail, inspired by the ALTAI framework discussed in Section 3, our proposed framework loosely builds upon ALTAI’s principles to offer a user-friendly tool. We were particularly drawn to ALTAI’s structured approach and its use of questions, which facilitates a clearer and more intuitive evaluation process. By adopting this format, we preserve the robustness of ALTAI’s criteria while transitioning to a more accessible list of questions that minimizes technical details. ALTAI’s thorough coverage of trustworthiness dimensions, such as transparency, technical robustness, and safety, underscores critical areas that we believe users should familiarize themselves with to effectively engage with AI applications and identify potential threats. By adapting and simplifying core principles from ALTAI, our framework aspires to raise user awareness but also integrate these dimensions into everyday discussions about AI, making the navigation and assessment of AI technologies more intuitive. The detailed mapping between the dimensions of the two frameworks is presented in

Table 2. Mapping of AI User Self-Assessment Framework dimensions to ALTAI dimensions.

ALTAI	AI User Self-Assessment Framework
Human Agency and Oversight	User Control
Transparency	Transparency
Technical Robustness and Safety	Security
Privacy and Data Governance	Privacy
Diversity, Non-discrimination, Fairness	Ethics
Accountability	Compliance
Societal and Environmental Well-being	Not explicitly mapped

.

As outlined in

Table 3. AI User Self-Assessment Framework: Example questions, criteria, and objectives for each dimension of trustworthiness assessment within the proposed framework. The framework’s dimensions are marked in bold. Detailed sets of questions for each dimension are elaborated in Appendix A.

	Example Questions	On-App Criteria	Objective
User Control	Do you feel like you can step in and change or override the AI app’s decisions?	Options to override AI decisions, emergency halt functions.	Ensure users maintain control over the AI app’s operations, promoting autonomy and the ability to intervene.
Transparency	Can you find out what data the app used to make a specific decision or recommendation?	Clear explanations of data sources and decision-making processes.	Promote understanding and clarity regarding the AI app’s actions and decisions, encouraging users to value transparency.
Security	Do you think that the app could cause serious safety issues for users or society if it breaks or gets hacked?	Robust data encryption, regular security audits, and proactive threat detection.	Ensure users feel confident about the app’s safety measures and its ability to protect their data and functionality.
Privacy	Do you feel that the app respects your privacy?	Transparent data collection policies, easy-to-access consent and data deletion options.	Empower users to make informed decisions about their personal information and understand how their data are handled.
Ethics	Do you think that the app creators have a plan to avoid creating or reinforcing unfair bias in how the app works?	Bias detection and correction mechanisms, adherence to ethical guidelines.	Ensure fairness, inclusivity, and alignment with ethical standards.
Compliance	Are there clear ways for you to report problems or concerns about the app?	Accessible reporting mechanisms, regular ethical reviews, and compliance with legal standards.	Empower users to hold the app accountable and ensure it adheres to relevant laws and ethical guidelines.

and further detailed in Appendix A, the proposed framework provides a comprehensive set of criteria and questions designed to guide users in their assessments, covering the following dimensions:

• User Control. This dimension focuses on users’ ability to understand and manage the AI app’s actions and decisions. It assesses whether users can control the app’s operations and intervene or override decisions if needed. The aim is to ensure users maintain their autonomy and that the AI app supports their decision-making without undermining it. Key areas include awareness of AI-driven decisions, control over the app, and measures to avoid over-reliance on the technology. A detailed set of questions enabling users to assess the dimension of user control is available in Appendix A.1.
• Transparency. The set of questions related to the dimension of transparency (as detailed in Appendix A.2), aim to enable users to assesses how openly the AI app operates. It focuses on the clarity and accessibility of information regarding the app’s data sources, decision-making processes, and purpose. It also evaluates how well the app communicates its benefits, limitations, and potential risks, ensuring users have a clear understanding of the AI’s operations.
• Security. This dimension aims to enable users assess the robustness of AI apps against potential threats, as well as mechanisms for handling security issues. The questions relevant to security, as detailed in Appendix A.3, evaluate how confident users feel about the app’s safety measures and ability to function without compromising data or causing harm. Important considerations include the app’s reliability, the use of high-quality information, and user awareness of security updates and risks.
• Privacy. This dimension examines how the AI app handles personal data, emphasizing data protection and user awareness of data use. It considers users’ ability to control their personal information and the app’s commitment to collecting only necessary data. Topics include respect for user privacy, transparency in data collection, and options for data deletion and consent withdrawal. The specific questions aiming to enable users evaluate the app with respect to their privacy are available in Appendix A.4.
• Ethics. A set of questions relevant to the dimension of ethics, as detailed in Appendix A.5, aims to address the ethical considerations of the AI app, such as avoiding unfair bias and ensuring inclusivity. It evaluates whether the app is designed to be accessible to all users and whether it includes mechanisms for reporting and correcting discrimination. Ethical design principles and fairness in data usage are central themes.
• Compliance. This dimension of the framework looks at the processes in place for addressing user concerns and ensuring the AI app adheres to relevant legal and ethical standards. Through targeted questions, as detailed in Appendix A.6, it evaluates the availability of reporting mechanisms, regular reviews of ethical guidelines, and ways for users to seek redress if negatively affected by the app. Ensuring compliance with legal frameworks and promoting accountability are key aspects.

This structured approach empowers users to make well-informed decisions about the AI applications they engage with, promoting a more proactive and educated interaction with AI.

4.2. Framework Development Process

We developed the AI User Self-Assessment Framework through a structured process:

• Literature Review. We began with a review of the existing literature on the threats posed by popular AI applications, using real-world incidents as case studies. This review highlighted how these threats undermine AI trustworthiness and how positive user experiences can sometimes obscure underlying trustworthiness issues. We also examined current countermeasures to these threats. Exploring existing solutions for assessing AI trustworthiness, we identified key dimensions and principles that became the foundation for the framework’s core elements.
• Framework Design. Based on the identified dimensions of AI Trustworthiness, we designed the framework by adapting principles from the ALTAI framework and incorporating user-centric elements. We simplified questions from the ALTAI framework to make them more relevant to users without technical expertise. This design process involved defining specific criteria and crafting user-friendly questions for each dimension. The aim was to create questions that users, regardless of their technical background, can understand and engage with the framework, improving the accessibility and effectiveness of the self-assessment tool.
• Framework Implementation. Following the design of the framework, we developed a practical self-assessment tool that incorporates all identified questions. The tool features a user-friendly interface where each dimension is represented by a set of relevant questions. Users can interact with the tool by answering these questions based on their experiences with the AI app they are assessing. Additionally, the tool includes a scoring system that quantifies user responses. Each question is associated with specific criteria and scoring options, allowing users to assign scores based on how well the AI app meets the standards for each dimension according to their perceptions. The scores are aggregated to generate an ruling trustworthiness rating for the app in question. The tool was developed to facilitate individual assessments and allow for comparative analysis across different AI applications. The self-assessment tool, the different dimensions, as well as a subset of the questions, are demonstrated in Figure 3.
• Framework Application. Users can access our interface, where they are presented with a list of popular AI applications, such as ChatGPT, for trustworthiness assessment. Upon selecting an app, users are guided through a series of questions covering the identified dimensions of AI trustworthiness. As they respond to the questions, their answers are scored based on predefined criteria for each dimension. Once the user completes the assessment, the tool aggregates their individual scores. These scores are then combined with assessments from other users evaluating the same AI app, providing a trustworthiness rating. This aggregated rating helps create a collective evaluation of the app’s trustworthiness.

4.3. User Scoring and Evaluation Process

The AI User Self-Assessment Framework incorporates a scoring system designed to evaluate AI applications based on various dimensions of trustworthiness. This system allows users to assess applications through a set of standardized questions, providing scores that reflect their experiences and perceptions.

In the AI User Self-Assessment Framework, scores are assigned on a scale from 1 to 5 for the majority of questions, where 1 represents a low level of trustworthiness and 5 indicates a high level of trustworthiness. For instance, if a user is evaluating the Transparency dimension and responds to the question, “Can you easily find out what data the app used to make a specific decision?” a score of 1 might be given if the app offers no information about its data sources, indicating a lack of transparency. A score of 3 would suggest that the app provides some basic information but lacks clarity or accessibility, while a score of 5 would signify that the app explicitly lists data sources and clearly explains its decision-making process. Additionally, the framework includes questions that allow users to respond with “yes” or “no” to evaluate specific features, such as whether they feel they are overly reliant on the app. This format provides users with a straightforward way to express their perceptions and experiences regarding the app.

The scoring system aggregates user inputs to provide a detailed assessment of an AI application’s trustworthiness. By evaluating various dimensions of trustworthiness, the system offers a holistic view of how an AI application performs in real-world scenarios. The analysis of these aggregated scores enables us to compare the perceived trustworthiness of an AI application with its perceived usability or user experience. This comparative analysis can help users understand how trustworthy an app is but also how it aligns with their expectations for usability and satisfaction.

The aggregated scores enable both individual assessments and a broader understanding of AI technologies’ perceptions across various contexts and user demographics. This approach promotes a feedback loop, where user evaluations inform improvements in AI development while also spotlighting areas that require attention.

5. Conclusions

In this work, we have introduced the AI User Self-Assessment Framework, a tool designed to empower users in evaluating the trustworthiness of popular AI applications based on their personal experiences. The framework’s integrated scoring system allows users to assess various dimensions of AI trustworthiness, providing insights into how these applications are perceived.

Looking ahead, we envision the integration of this framework into collaborative rating systems and community-driven platforms. Future developments may include a public leaderboard that ranks AI applications according to their trustworthiness. This leaderboard would aggregate user scores, creating a competitive incentive for developers to adhere to higher standards and address user concerns more effectively. Additionally, future research will explore the combination of the trustworthiness score with the System Usability Scale (SUS) [59] to investigate how usability scores might shift when weighted by trustworthiness factors, offering a more holistic view of AI app performance.

To fully realize the potential of the framework, addressing key challenges such as optimizing the relevance of questions, maintaining consistency in scoring, and incorporating continuous updates will be essential. Integrating insights from diverse fields—including computer science, ethics, law, and social sciences—will be crucial in advancing AI accountability and ensuring that these technologies meet high standards of trustworthiness.