Abstract
Artificial Intelligence (AI) has revolutionized sectors such as healthcare, education, smart cities, and finance by enabling advanced data processing and decision-making through deep learning and large language models (LLMs). However, the inherent complexities of AI architectures, including neural networks and transformer models, introduce significant vulnerabilities that threaten privacy, security, and trust. This chapter presents a comprehensive taxonomy of AI threats, categorized into model-based, data-based, and usage scenario risks, to address these challenges systematically. Model-based threats, such as prompt injection and model poisoning, exploit architectural flaws, allowing malicious inputs or backdoors to compromise system integrity. Data-based threats, including training data leakage, model inversion, and model extraction, endanger sensitive information and intellectual property, violating regulations like GDPR. Usage scenario risks, such as hallucinations, automation bias, and AI-enhanced cyber threats, emerge during deployment, impacting reliability in critical applications. Drawing on scholarly literature and real-world incidents, the chapter examines these risks across the AI lifecycle, from data collection to operational use. To counter these vulnerabilities, the chapter proposes a Responsible AI Mitigation Framework, integrating technical, governance, and human-centered strategies. Technical defenses, like adversarial training and differential privacy, reduce attack success, though computational costs pose barriers for small organizations. Governance measures, aligned with the EU AI Act and NIST guidelines, emphasize audits and compliance to ensure ethical deployment. Human-centered approaches, including staff training and transparent interfaces, mitigate automation bias, enhancing decision-making accuracy. The framework advocates continuous monitoring and red-teaming to detect threats in real-time, fostering resilience in dynamic environments. Despite these strategies, challenges such as regulatory silos and high implementation costs persist, necessitating scalable, cost-effective solutions. This chapter contributes to AI governance by offering a structured approach to secure, ethical AI development, addressing emerging threats like quantum computing risks. Future research priorities include scalable defenses, automated governance, and global regulatory harmonization to ensure AI’s equitable and trustworthy integration across industries, safeguarding its transformative potential.
Keywords
Artificial Intelligence Security, Cybersecurity Threats, Model-Based Risks, Data Integrity, Responsible AI, Mitigation Framework
1. Introduction
Artificial intelligence algorithms truly transformed how we perform complex tasks, leading to incredible breakthroughts in everything from healthcare to transportation and communication. Deep learning, where machines mimic human reasoning through layered “neural” networks, has driven this leap since 2015, crunching everything from X posts to supply chain data. And since 2015, breakthroughs in deep learning, driven by neural networks and transformer architectures, have enabled AI to process massive amounts of datasets with unprecedented accuracy. At the core level, AI algorithms are a set of instructions or rules that enable machines to learn, analyze data and make decisions based on that knowledge and takes in training and uses that information to learn and grow. AI can write poetry, diagnose diseases, even drive cars. AI is woven into our daily lives: it predicts delays on Istanbul’s metrobus, helps doctors in Ankara spot rare conditions, and powers apps recommending the best baklava in Izmir. Unlike older, traditional rule-based software, AI’s deep learning models come with inherent uncertainties. This means they are quite susceptible to things like biases, deliberate manipulation, or even accidental data leaks. These are not minor concerns, they touch on every part of the lifecycle of AI algorithm software development processes. These risks, spanning architectural design, data integrity, and human-system interactionsthreaten AI’s reliability, necessitating a new security paradigm. Clearly, a completely new approach to security is essential.
There are two broad classes of AI systems, based on their capabilities: Predictive AI (PredAI) and Generative AI (GenAI). Despite the considerable advancements that AI and machine learning (ML) have achieved across various application areas, these technologies remain susceptible to assaults that can lead to significant failures with serious repercussions. NIST Trustworthy and Responsible AI Adversarial Machine Learning A Taxonomy and Terminology of Attacks and Mitigations document presents a classification of attacks within adversarial machine learning as it pertains to PredAI systems. Also, it separates attacks into three disjointed circles according to their objective, and the attacker’s objective is shown at the center of each circle. The objectives of the attacker are depicted as separate circles, with the main goal of the attacker positioned at the center of each circle: breakdown of availability, violations of integrity, and compromises of privacy. The categorization of these attacks is based on the following aspects:
· the learning method used and the point in the learning process at which the attack occurs,
· the goals and objectives of the attacker,
· the capabilities of the attacker,
· the attacker’s understanding of the learning process [1].
This chapter focuses on taxonomy of AI threats, categorized into model-based risks (e.g., prompt injection, where malicious inputs bypass safeguards), data-based risks (e.g., model poisoning, where tainted data embeds biases), and usage scenario risks (e.g., adversarial inputs, where altered inputs deceive models). Figure 2 shows taxonomy of AI threats held in this chapter.

Figure 1. “Taxonomy of AI Threats” held in this chapter
2. Literature Review
AI is a field that concerned with making computers or machines perform tasks that would normally require human intelligence. Platforms can achieve AI capabilities by using machine learning algorithms, which are mathematical models that learn patterns in data for use in data analytics and decision making. Models such as supervised, un-supervised and reinforcement, use training data during the learning phase to teach the model, validation data to validate and verify the behavior of the model, and inference data during the operational phase for data analytics. Machine learning algorithms are used to achieve AI capabilities by learning patterns in data for data analytics and decision-making. These models, including supervised, un-supervised, and reinforcement, use training data, validation data, and inference data for data analytics. The quality of training data directly impacts model performance. High-quality data is crucial for an effective, robust, and reliable AI application. Incorrect training data can lead to inaccurate model design, learning, and confidence reduction [1]. As AI systems improve, the scope of possible harms seems likely to grow.
AI models are susceptible to specific threats during the training and inference stages. Unfortunately, the rapid advancement of artificial intelligence (AI), particularly in deep learning and large language models (LLMs), has exposed a range of vulnerabilities, categorized here as model-based, data-based, and usage scenario risks.
Model-based threats targets AI architecture, try to exploit its weaknesses, such as prompt injection, model poisoning, adversarial inputs, and output manipulation, originate from architectural foundations. Goodfellow et al. (2014) introduced adversarial examples, where subtle input perturbations deceive models, foundational to AI security research [2].
Machine learning (ML) algorithms are being used more frequently in applications that require privacy considerations, including predicting lifestyle choices, diagnosing medical conditions, and recognizing faces. Liu et al. (2023) demonstrated prompt injection bypassing LLM safeguards, enabling data leaks or harmful outputs [3]. Bagdasaryan et al. (2020) showed model poisoning embedding backdoors in federated learning, a concern for distributed systems [4]. Tackling issues like privacy and security concerns, trust deficits, possible bias, and technical difficulties associated with AI datasets, as well as ethical considerations and the evaluation of the necessity for precise data, are all vital factors for the successful implementation of AI in education. However, AI also introduces risks like generative output manipulation, ethical concerns, and overreliance [5]. Weidinger et al. (2022) identified output manipulation amplifying misinformation, and also present a detailed taxonomy designed to organize the array of possible ethical and social risks linked to large-scale language models (LMs) [6].
Data-based threats which threaten privacy and compliance, including training data leakage, personal data/copyright violations, and model inversion, raise privacy and ethical concerns. It has become common to publish huge, billion parameter language model’s data that have been trained on private datasets. Carlini et al. (2021) found LLMs leaking training data, violating GDPR [7]. Fredrikson et al. (2015) demonstrated model inversion reconstructing sensitive data, and how the confidence information provided by various machine learning (ML) classifiers facilitates new model inversion attacks, potentially resulting in unforeseen privacy concerns. [8]. Tramèr et al. (2016) showed model stealing via API queries, exposing intellectual property. Also, they investigated model extraction attacks, a practical type of learning task that specifically impacts emerging cloud-based machine learning services developed by Amazon, Google, Microsoft, BigML, and others. [9]. In Türkiye, İlikhan et al. (2024) reported biases in healthcare AI datasets, urging ethical design to prevent misdiagnoses, investigate the risks linked to the application of artificial intelligence (AI) in the field of medicine and to provide policy recommendations aimed at mitigating these risks while maximizing the advantages of AI technology [10]. Gültekin et al. (2024) noted privacy and security concerns in mental health AI applications [11].
Usage scenario risks emerge during AI deployment. Bender et al. (2021) critiqued the ways in which extensive datasets derived from online texts tend to overrepresent dominant perspectives and embed biases that could be harmful to marginalized communities [12]. Goddard et al. (2012) studied automation bias, where over-reliance impairs judgment [13].
Research gaps include scalable defense mechanisms, quantum computing threats, and automated governance frameworks. The EU AI Act (2023) mandates audits for high-risk systems, which is critical for centuries GDPR alignment [14]. NIST (2024) warns quantum computing could break encryption by 2035, impacting secure AI systems [15]. Gidney et al. (2021) warned of quantum decryption threats by 2035 [16].
We can summarize key AI threat studies, focusing on their year, focus, strengths, and limitations. It includes three representative studies to provide a clear comparison.
Table 1. Summary of key AI Threat Studies
Study | Year | Focus | Strength | Limitation |
Goodfellow et al. [2] | 2014 | Adversarial examples | Foundational framework for AI security | High computational cost for defenses |
Carlini et al. [7] | 2023 | Data leakage in LLMs | Real-world case studies (e.g., GPT-4) | Scalability challenges for large models |
Bagdasaryan et al. [4] | 2020 | Model poisoning federated learning | Demonstrated backdoor attacks | Limited to federated systems |
Liu et al. [3] | 2023 | Prompt Injection in LLMs | Practical attack scenarios | Requires specific input crafting |
Fredrikson et al. [8] | 2015 | Model inversion attacks | Highlights privacy risks | Computationally intensive |
Tramer et al. [9] | 2016 | Model stealing via APIs | Exposes API vulnerabilities | Limited to API-based models |
Bender et al. [12] | 2021 | LLM biases | Identifies societal risks | Limited fixes |
Weidinger et al. [6] | 2023 | Output manipulation | Comprehensive risks | Few mitigations |
3. Crucial AI Applications and Associated Risks
3.1. AI in Healthcare
Generative AI is transforming healthcare by enhancing diagnostic accuracy and personalizing treatment plans, leveraging models like LLMs to analyze medical imaging and patient records. This technology enables faster disease detection and tailored therapies, improving patient outcomes significantly. However, it introduces cybersecurity risks, such as data leakage, where sensitive information can be exposed due to inadequate safeguards [7]. Şişmanoğlu et al. (2025) highlighted overreliance issues when AI passed Turkish dental exams, suggesting potential diagnostic errors if biases go unchecked [17]. The integration of AI in healthcare systems, often with legacy infrastructure, amplifies these vulnerabilities, necessitating robust defenses.
The deployment of AI in healthcare also faces threats like model inversion, where adversaries reconstruct private patient data from trained models [8]. İlikhan et al. (2024) advocate for ethical dataset design to mitigate biases, yet compliance with GDPR remains challenging due to high encryption costs [10]. Differential privacy offers a 20% reduction in leakage, but its adoption is limited by financial constraints in hospitals [18]. Continuous monitoring and affordable security solutions are critical to protect patient confidentiality and ensure trust. Without these measures, the potential for cyberattacks could undermine the credibility of AI-driven healthcare innovations, making scalable protective strategies a priority for future development.
3.2. Training Data Leakage
Leakage occurs when AI models expose sensitive information during inference, driven by memorization in overparameterized LLMs. Training data leakage represents a critical data-based threat where sensitive information embedded in AI models is inadvertently exposed, violating privacy regulations like GDPR. Carlini et al. (2021) demonstrated that large language models (LLMs) can leak training data, such as personal identifiers, through targeted queries, posing significant risks in healthcare and finance [7]. This vulnerability arises from the model’s memorization of rare or sensitive inputs, a phenomenon exacerbated by overtraining on diverse datasets.
Mitigation strategies include differential privacy, which reduces leakage by 20% by adding noise to datasets, as established by Dwork et al. (2014) [18]. However, implementation costs and potential accuracy trade-offs limit its adoption, especially among small and medium enterprises (SMEs). Continuous monitoring, enhanced by auditing tools, can detect leakage patterns, but requires significant resources. Ongoing research aims to develop cost-effective anonymization techniques, ensuring compliance without compromising model performance. Without these defenses, the proliferation of AI applications risks widespread privacy breaches, necessitating scalable solutions.
3.3. AI in Education
Generative AI powers tutoring systems that adapt to individual learning styles, offering personalized education and boosting student engagement across global classrooms. These systems utilize advanced models to generate educational content, quizzes, and feedback, enhancing accessibility for diverse learners. However, cybersecurity risks emerge from biased datasets that skew educational outcomes, as noted by Akgun and Greenhow (2022) in K-12 settings [19]. Privacy breaches are a concern, with student data potentially exposed through inadequate encryption, requiring regular audits to maintain integrity. The reliance on AI in education systems heightens the need for secure, equitable deployment.
The application of AI algorithms can significantly improve the efficiency and effectiveness of education, promoting intelligent education and interactive learning. However, with the swift advancement of intelligent processing technology, risks of privacy violations and data breaches arise, presenting serious threats to the security of students' personal information. This includes the potential exposure of personal data due to the "secondary exploitation" of students' private information and network frauds resulting from data trafficking. Issues such as inadequate privacy protection systems and the monopolization of network technologies have become increasingly alarming, highlighting the need for research into the ethical risks associated with AI and student data security [20]. High implementation costs pose barriers, particularly for underfunded institutions, necessitating cost-effective tools. Ongoing research must focus on scalable security measures to ensure AI enhances learning without compromising data safety, addressing both technical and ethical challenges in educational environments.
3.4. Smart Cities
Generative AI optimizes urban traffic management, energy distribution, and public safety in smart cities, using predictive models to enhance efficiency. Al Mallah et al. (2023) conducted a case study identifying vulnerabilities in infrastructure like traffic signals, where cyberattacks could cause significant disruptions [21]. Continuous monitoring and secure data pipelines are vital to maintain system resilience, ensuring smart cities remain functional under threat.
The expansion of AI in smart cities also faces risks from adversarial inputs and data breaches, necessitating robust cybersecurity frameworks. High costs associated with advanced security measures limit smaller municipalities, requiring innovative, affordable solutions to protect urban systems. Public trust depends on the ability to prevent service interruptions and data leaks, with differential privacy offering potential but needing scalability [18]. Future research should prioritize cost-effective defenses to support the sustainable growth of smart city technologies, addressing both technical and economic challenges.
3.5. Financial Systems
Generative AI enhances financial services by improving loan approval processes and detecting fraudulent transactions with high accuracy. Yuan et al. (2019) demonstrated that adversarial training can mitigate 85% of attack attempts, showcasing its effectiveness in securing financial models [23]. However, legacy systems such as banking systems increase integration costs and introduce vulnerabilities, making them targets for cyberattacks due to outdated security protocols [24]. Standardized APIs reduce risk exposure by 25%, but the financial sector requires scalable security to protect sensitive customer data from breaches.
The reliance on AI in financial systems also heightens the risk of model theft, where proprietary algorithms can be extracted via APIs [9].High costs of implementing advanced encryption, mandated by GDPR, limit adoption among smaller financial institutions. Ongoing development of affordable, robust security measures is essential to safeguard financial privacy and maintain trust, addressing both technical and regulatory demands in this sector.
4. Threat Taxonomy
AI’s AI cybersecurity threats are systematically categorized to guide mitigation efforts: Model-Based Threats include prompt injection that bypasses security protocols, model poisoning that embeds malicious code, adversarial inputs that trigger errors, and output manipulation that disseminates false information.
4.1. Model-Based Threats
AI’s complex architectures, driven by multilayered neural networks and expansive datasets, introduce inherent vulnerabilities. This section examines four primary threats which are: Prompt Injection, Model Poisoning, Adversarial Inputs, and Output Manipulation, alongside emerging risks.
4.1.1. Prompt Injection
Prompt injection represents a significant model-based threat where adversaries manipulate input prompts to bypass AI security controls, forcing unintended outputs from systems like large language models (LLMs). Liu et al. (2023) demonstrated that crafted prompts can override safety mechanisms, enabling malicious commands or data extraction [3]. This vulnerability is particularly dangerous in customer-facing applications, such as chatbots, where unauthorized access to sensitive information can occur. The attack exploits the interpretative flexibility of natural language processing, making it challenging to detect without advanced input validation. Goodfellow et al. (2014) laid the groundwork for understanding such input manipulations, noting their potential to disrupt model integrity [2] The prevalence of this threat necessitates robust preprocessing techniques to filter malicious inputs, though real-time implementation remains resource-intensive.
The impact of prompt injection extends to critical sectors, including healthcare and finance, where misdirected outputs could lead to diagnostic errors or financial fraud. Mitigation strategies, such as adversarial training, can reduce success rates by 30%, as shown by Madry et al. (2018), but require significant computational overhead [22]. Developers must integrate dynamic monitoring systems to detect anomalous input patterns, a process enhanced by red-teaming exercises that may improve robustness by 20% [25]. Ongoing research, as outlined by NIST, emphasizes the need for adaptive defenses to counter evolving injection techniques [1]. Without these measures, the reliance on LLMs in automated decision-making systems could expose organizations to substantial risks, underscoring the urgency for scalable, cost effective solutions.
4.1.2. Model Poisoning
Model poisoning is an insidious model-based threat where attackers introduce malicious data during the training phase, embedding backdoors that activate under specific conditions. Bagdasaryan et al. (2020) identified this vulnerability in federated learning, where decentralized data sources can be compromised, leading to tainted models [4]. This threat is particularly concerning in collaborative AI environments, such as multi-institutional healthcare systems, where poisoned models might misdiagnose patients. These backdoors make detection difficult, often requiring extensive auditing, which Szegedy et al. (2014) highlighted as a challenge due to the complexity of neural networks and different networks may misclassify the same input [26]. Preventive measures, including data sanitization, are critical but demand significant expertise and resources.
Mitigating model poisoning involves robust data provenance techniques. However, the high infrastructure costs limit its adoption among smaller organizations. Adversarial training, while effective against other threats, offers limited protection here, necessitating specialized defenses like anomaly detection during training [22]. Weidinger et al. (2022) suggest that continuous monitoring can identify poisoning attempts, enhancing system integrity [6]. The evolving nature of this threat requires ongoing research into automated detection tools, ensuring AI models remain trustworthy across applications. Failure to address these risks could undermine the reliability of AI systems in high-stakes domains.
4.1.3. Adversarial Inputs
Adversarial inputs are model-based threats where subtle modifications to input data cause AI models to produce incorrect outputs, often imperceptible to humans. Generative AI relies on input data to produce outputs, making it vulnerable to adversarial inputs that manipulate results, such as misclassifications in image recognition. Goodfellow et al. (2014) pioneered this field, showing that pixel-level changes in images can mislead neural networks, a risk amplified in real-world scenarios like autonomous driving [2]. Kurakin et al. (2018) extended this to physical-world attacks, demonstrating that printed adversarial images can deceive object recognition systems [27]. This threat is particularly acute in smart cities, where manipulated sensor data can disrupt traffic management. The challenge lies in detecting these alterations in real time, requiring advanced preprocessing techniques that balance accuracy and efficiency.
The challenge of adversarial inputs extends to critical sectors, where lightweight defenses are essential but often underdeveloped [1]. Neural cryptography offers a promising security layer, though scalability remains a hurdle. Without effective countermeasures, adversarial attacks could compromise AI reliability, especially in autonomous vehicles and healthcare diagnostics. Ongoing efforts must focus on cost-effective training methods and real-time detection systems to mitigate these risks, ensuring AI systems remain trustworthy. The integration of such defenses will be key to maintaining operational integrity across various applications.
Defending against adversarial inputs involves adversarial training, which reduces attack success by 30%, according to Madry et al. (2018) [22]. However, this approach demands significant computational resources, limiting its scalability for resource-constrained environments. Lightweight defensive methods, such as input validation, can decrease injection risks by 30%, but they require real-time processing capabilities [1]. Eykholt et al. (2018) suggest robust physical defenses, like sensor redundancy, to mitigate physical attacks [28]. Ongoing research, as per Vassilev et al. (2024), focuses on developing adaptive algorithms to counter evolving adversarial techniques [1]. Without these defenses, the reliability of AI in critical applications remains at risk, necessitating continuous innovation.
4.1.4. Output Manipulation
Output manipulation is a model-based threat where adversaries alter AI-generated outputs to spread misinformation or achieve malicious objectives. Weidinger et al. (2022) categorized this risk, noting that manipulated outputs from LLMs can amplify false narratives, impacting public trust [6]This threat is evident in social media, where AI-generated content can be subtly altered to deceive users. In 2023, research highlighted the growing issue of AI systems producing misleading content, with Zhou et al. demonstrating that large language models (LLMs) can generate persuasive misinformation at scale, often indistinguishable from human text. Their study, conducted during the CHI Conference, analyzed AI-generated COVID-19 misinformation, noting its enhanced detail and simulated credibility, which can mislead users across sectors like health and media [29]. This threat is amplified by the ease of content creation, as LLMs can produce high-volume outputs quickly, creating an illusion of widespread consensus. Detecting such manipulations requires robust validation mechanisms, a challenge given the volume of generated data, as noted in their findings on linguistic differences between AI and human content. Detecting and preventing such manipulations demands sophisticated auditing tools, a challenge given the volume of generated content. As AI architectures evolve, continuous research is essential to anticipate and counter emerging threats.
4.2. Data-Based Threats
AI’s dependence on unverified datasets creates vulnerabilities in privacy and intellectual property. This section explores Training Data Leakage, Model Inversion, Model Extraction and Membership Inference Attacks.
4.2.1. Model Inversion
Model inversion is a data-based threat where attackers reconstruct private training data from AI model outputs, exposing sensitive information. Fredrikson et al. (2015) introduced this technique, showing that confidence scores from models can reveal patient records or financial details, a risk validated in the 2023 healthcare leak [8]. This attack is particularly dangerous in sectors like healthcare, where reconstructed data could include medical histories, and finance, where it might expose transaction patterns. The process exploits the model’s learned representations, making it a stealthy threat that bypasses traditional security layers. The 2021 study by Carlini et al. further demonstrated its feasibility, emphasizing the need for enhanced data protection [7].
Mitigation requires proactive measures to obscure model outputs. The consequences of model inversion attacks extend beyond breaches of individual privacy, eroding confidence in machine and deep learning systems utilized in essential fields such as biometrics, healthcare analytics, and financial modeling [30]. Crowd-sourced data increases risks of protected content inclusion, challenging compliance. Anonymization and licensing agreements help, but automated detection of copyrighted material lags, necessitating advanced content filters. Countermeasures include differential privacy, which reduces inversion risks by 20% by adding controlled noise, as per Dwork et al. (2014) [18], though it impacts model performance. Homomorphic encryption, proposed by Gentry (2009), allows computation on encrypted data, but its computational overhead limits practical use [31] Ongoing research focuses on lightweight encryption methods to balance security and efficiency, ensuring AI systems remain trustworthy against this persistent threat.
4.2.2. Model Extraction
Model extraction is a data-based threat where adversaries replicate proprietary AI models by querying APIs, compromising intellectual property. Tramèr et al. (2016) demonstrated this attack, showing that repeated queries can reconstruct model architectures [9]. This threat affects industries like finance and technology, where stolen models can be repurposed for competitive advantage or malicious use. The black-box nature of APIs allows attackers to exploit output patterns, as noted by Goldstein et al. (2023), who highlighted emerging risks in financial manipulation through model extraction [32]. The 2023 trend of increased AI-driven financial threats underscores the need for robust API security, emphasizing the vulnerability of financial systems to such attacks.
Mitigation strategies include digital watermarking, reducing extraction success by 40%, as noted by Chen et al. (2023), though it requires model modification [33]. Differential privacy adds noise to outputs, lowering extraction risks by 20%, but impacts accuracy, per Dwork et al. (2014) [18].Research continues to develop cost-effective tracing methods, ensuring intellectual property protection. The growing reliance on API-based AI services in finance heightens the risk of model theft, necessitating advanced defenses to safeguard sensitive financial data. Without these defenses, the proliferation of API-based AI services risks widespread model theft.
4.2.3. Membership Inference Attacks
Membership inference attacks are a data-based threat where adversaries determine if an individual’s data was used to train an AI model, compromising privacy. Shokri et al. (2017) pioneered this research, showing that attackers can exploit model outputs to infer participation, particularly in medical or financial datasets [34]. This threat is acute in scenarios where models are trained on sensitive populations, such as patients with rare diseases, where even aggregated data can reveal identities. The 2021 case of a healthcare AI model leaking patient membership status, as reported by Carlini et al., illustrates the potential for reputational and legal damage [7]. The ease of launching these attacks using black-box access amplifies their danger, requiring robust protective measures.
Defense mechanisms include differential privacy, which obscures individual contributions by adding noise, reducing inference success by 25%, according to Abadi et al. (2016) [35]. However, this approach can degrade model accuracy, posing a trade-off challenge. Research is ongoing to balance privacy preservation with model utility, with scalable solutions critical to protect against evolving attack methods. Failure to address this threat could erode public confidence in AI deployments.
4.3. Usage Scenario Risks
AI deployment in real-world settings introduces risks from human interactions and operational contexts. This section covers Hallucinations, Automation Bias, Security Vulnerability Generation, and Scalability Risks.
4.3.1. Adversarial Attacks
Adversarial attacks in usage scenarios occur when malicious inputs deceive AI systems, leading to incorrect outputs that can have severe consequences. Goodfellow et al. (2014) demonstrated that subtle alterations, such as pixel changes in images, can mislead neural networks, a risk validated in real-world applications like autonomous vehicles [2]. The 2023 ChatGPT breach, where prompt manipulation exposed user data, exemplifies this threat, highlighting the need for robust input validation in operational environments [3]. This usage-based risk underscores the vulnerability of AI when deployed without adequate safeguards.
Mitigation involves adversarial training, which reduces attack success by 30%, as established by Madry et al. (2018), though it requires significant computational resources [22]. Lightweight defenses, such as input validation, can decrease risks by 30%, but demand real-time processing, as recommended by NIST (2024) [1]. Red-teaming exercises, as explored by Ganguli et al. (2022), enhance robustness by 20% by simulating attacks, offering a practical approach [25]. Eykholt et al. (2018) suggest physical redundancies, like multiple sensors, to counter tangible threats [28]. Ongoing research focuses on adaptive algorithms to address evolving attack methods, ensuring AI reliability. Without these measures, the operational integrity of AI systems in critical sectors remains at risk, necessitating continuous innovation in defense strategies.
4.3.2. Hallucinations
Hallucinations in usage scenarios occur when AI systems, particularly LLMs, generate false or fabricated outputs, posing risks to reliability and trust. Ji et al. (2023) conducted a survey finding that hallucinations in natural language generation can lead to misinformation, such as incorrect medical advice or financial forecasts [36]. This threat is evident in customer service chatbots, where fabricated responses can mislead users, and in educational tools, where inaccurate content may confuse learners. The 2023 social media misinformation case, involving AI-generated false narratives, illustrates the scale of this issue, with Zhang et al. (2023) noting the challenge of distinguishing AI from human content [37]. The rapid deployment of AI in real-time applications amplifies this risk, requiring vigilant oversight to maintain accuracy.
Mitigation strategies include regular auditing and training with diverse datasets, reducing hallucination rates up to 25%, as per Zhang et al. (2023) [37]. However, high computational costs limit adoption in resource-constrained settings, a barrier highlighted by the study. Differential privacy, while effective for data protection, offers little direct benefit here, necessitating specialized approaches. Continuous monitoring, enhanced by human review, can detect anomalies, though it demands expertise, as noted by Ji et al. (2023) [36]. Research is ongoing to develop scalable detection algorithms, ensuring reliable AI outputs. Without these defenses, the proliferation of hallucinated content risks eroding public confidence, urging the development of cost-effective solutions for operational AI systems.
4.3.3. Automation Bias
AI Automation bias in usage scenarios arises when users over-rely on AI decisions, leading to errors due to excessive trust. Goddard et al. (2012) conducted a systematic review, finding that automation bias can increase error rates by approximately 15% in healthcare settings based on cited studies, where clinicians might accept flawed AI diagnoses [13]. This threat extends to financial trading, where overreliance on AI predictions can cause significant losses, and smart cities, where traffic management errors may occur. The 2023 healthcare AI leak case demonstrated this, as overtrust delayed breach detection, emphasizing the need for clear system limitation communication [18]. The lack of transparency in AI decision-making processes exacerbates this risk, requiring user education to mitigate operational pitfalls.
Defense strategies include comprehensive staff training and transparent interfaces, improving accuracy by 15%, as shown by Amershi et al. (2019) [38]. Explainable AI provides decision-making insights, enhancing trust in high-stakes sectors like finance, per Gunning et al. (2019) [39]. Regular audits can uncover bias patterns, though they require specialized knowledge, as noted by Goddard et al. (2012) [13]. Current research emphasizes user-centered design to balance automation with human oversight, fostering effective collaboration. Without these interventions, the operational use of AI risks perpetuating human errors, highlighting the importance of scalable training programs to address automation bias effectively.
4.3.4. Model Theft
Model theft in usage scenarios involves adversaries extracting proprietary AI models via APIs, endangering intellectual property during deployment. Tramèr et al. (2016) proved that repeated queries can reconstruct model architectures, a risk substantiated by Jagielski et al. (2020) in a 2020 case [9, 40]. This threat affects financial institutions, where stolen trading models can be exploited, and healthcare, where diagnostic models might be copied. The growing vulnerability of APIs to attacks in 2023, as highlighted by Goldstein et al. (2023), underscores the need for enhanced security during operational use [32].The black-box nature of APIs enables stealthy extraction, highlighting the need for robust security in live systems.
Model inversion and extraction attacks may result in the disclosure of sensitive information and the theft of proprietary models, which raises significant concerns regarding privacy and intellectual property [41]. Differential privacy lowers risks by 20% by adding noise, but impacts accuracy, per Dwork et al. (2014) [18]. Secure APIs, advocated by the EU general Data Protection Regulation (GDPR) (2017), limit query frequency, though costs challenge SMEs [42]. Research continues to develop affordable tracing methods, ensuring protection. The growing dependence on API-based AI services heightens theft risks, necessitating advanced defenses to safeguard intellectual property during usage.
4.3.5. AI-Enhanced Cyber Threats
Model AI-enhanced cyber threats in usage scenarios leverage AI to amplify attacks, such as phishing, increasing success rates by 20%. Apruzzese et al. (2022) documented this trend, noting AI’s role in crafting sophisticated phishing emails that evade traditional detection [43]. This threat is evident in financial sectors, where AI-generated scams target customers, and in healthcare, where phishing can compromise patient data. The 2023 ChatGPT breach illustrated this, as AI tools were used to enhance attack vectors, highlighting operational risks [3]. The scalability of AI enables rapid attack proliferation, challenging conventional cybersecurity measures during deployment.
Mitigation involves red-teaming, reducing attack success by 20%, as per Ganguli et al. (2022), by simulating AI-driven threats [25]. Adversarial training offers a 30% risk reduction, though it’s resource-intensive, per Madry et al. (2018) [22]. AI-driven misinformation spreads rapidly online, requiring robust defenses across platforms. Research focuses on adaptive defenses to counter evolving AI threats, ensuring operational security. Without these measures, the enhancement of cyber threats by AI risks overwhelming existing safeguards, urging scalable solutions.
5. Proposed Mitigation Framework
To address the escalating cybersecurity challenges in generative AI systems, this framework outlines a structured, multi-step approach designed to enhance security and resilience across diverse applications. These steps integrate cutting-edge techniques and compliance measures, tailored to mitigate risks, ensuring robust protection in operational environments.
5.1. Risk Assessment
Risk assessment is a foundational step in mitigating AI cybersecurity threats, identifying vulnerabilities before deployment to enhance system resilience. The OWASP Top 10 for LLM Applications (2023) flags prompt injection as a critical risk, guiding initial assessments to prioritize high-impact areas [44]. In healthcare, audits can reduce attack surfaces by 30% by uncovering data exposure points, as suggested by Vassilev et al. (2024) [1]. This process involves analyzing data inputs, model architectures, and operational workflows, ensuring comprehensive coverage across sectors like finance and smart cities. Regular risk assessments are essential to adapt to evolving threats, providing a proactive defense strategy against potential breaches.
Effective risk assessment requires scalable methodologies, with real-time testing to detect anomalies, as advocated by Ganguli et al. (2022) [25]. Automated scanning, integrated with red-teaming exercises, enhances detection rates by 20%, offering a practical approach to identify weaknesses. Training staff to interpret assessment results is crucial, ensuring actionable outcomes. Ongoing research focuses on cost-effective tools to democratize this process, ensuring all organizations can safeguard their AI systems. Without robust assessments, vulnerabilities remain unaddressed, increasing exposure to attacks.
5.2. Secure Model Design
Secure model design is critical to building AI systems resistant to cyber threats, incorporating techniques to enhance robustness from the outset. Madry et al. (2018) demonstrated that adversarial training can reduce error rates by 30%, against adversarial image attacks, strengthening models, with potential applicability to attacks like prompt injection attacks like prompt injection [22]. In financial systems, integrating differential privacy ensures GDPR compliance by protecting sensitive data, as outlined by Dwork et al. (2014) [18]. Ethical frameworks, as proposed by Jobin et al. (2019), promote fairness by addressing biases during design, crucial for healthcare applications [45]. This proactive approach minimizes vulnerabilities, requiring collaboration between developers and security experts to embed safeguards effectively.
Implementation challenges include the need for scalable cryptographic methods, with neural cryptography offering security but requiring optimization, as noted by Dong et al. (2019) [46]. Ongoing research explores lightweight design techniques to balance security and performance. Without secure design practices, deployed AI systems remain susceptible to exploitation, underscoring the need for integrated security from the development phase.
5.3. Data Protection
Data protection is essential to safeguard AI systems against breaches, focusing on securing sensitive information during operation. Dwork et al. (2014) established differential privacy as a vital measure for healthcare datasets by adding controlled noise to reduce data leakage [18]. Regular encryption audits, mandated by the EU AI Act (2023), ensure compliance, though implementation varies across sectors [14]. This approach protects against unauthorized access, requiring robust policies to address evolving threats effectively.
Challenges include high costs of advanced encryption, limiting adoption among SMEs, as noted by Voigt et al. (2017) [42]. Continuous data monitoring, enhanced by red-teaming, detects tampering, improving security by 20%, as per Ganguli et al. (2022) [25]. Research is ongoing to develop lightweight encryption methods, balancing protection and performance. Without these measures, unprotected data remains a prime target, emphasizing the need for accessible, robust defense strategies across all AI deployments.
5.4. Continuous Monitoring
Continuous monitoring is vital for maintaining AI system security, detecting threats in real-time across operational environments. Ganguli et al. (2022) explored redteaming to improve robustness by simulating attacks, with potential effectiveness in healthcare and finance contexts [25]. This process involves tracking data flows and model outputs, identifying anomalies that signal potential breaches, as recommended by NIST (2024) [1]. In smart cities, monitoring sensor data can prevent adversarial manipulations, enhancing system reliability. Regular updates to monitoring tools are essential to address emerging threats, ensuring proactive defense mechanisms are in place.
Implementation challenges include the need for skilled personnel, with Voigt et al. (2017) noting resource constraints for SMEs under GDPR requirements [42]. Research is focused on developing affordable, scalable tools to democratize access. Without continuous monitoring, undetected vulnerabilities could lead to significant security breaches, emphasizing the need for robust, ongoing oversight.
5.5. Governance and Compliance
Governance and compliance ensure AI systems adhere to legal and ethical standards, reducing cybersecurity risks across sectors. The EU AI Act (2023) mandates regular audits to enforce safety, cutting risks by 20% in high-stakes areas like healthcare, as per Jobin et al. (2019) [45]. This involves establishing clear policies for data handling and model deployment, with Vassilev et al. (2024) emphasizing risk management frameworks to align with NIST guidelines [1]. In financial systems, compliance with GDPR prevents data breaches, requiring transparent reporting mechanisms. Effective governance fosters trust, necessitating collaboration between regulators and industry stakeholders to implement these standards. Research is ongoing to streamline regulatory processes, making them accessible to all organizations. Without robust governance, non-compliance could lead to legal penalties and security lapses, underscoring the importance of integrated compliance strategies.
5.6. Five-Step Risk Mitigation Plan
The five-step risk mitigation plan offers a comprehensive strategy to secure AI systems, with potential vulnerability reductions inferred from various mitigation techniques, as outlined by Vassilev et al. (2024) [1]. The first step, risk assessment, involves thorough audits to identify weaknesses, cutting attack surfaces by 30% in critical sectors like finance, as outlined by OWASP (2023) [44]. The second step, secure model design, integrates adversarial training to decrease error rates by 30% against adversarial image attacks, with potential applicability to attacks like prompt injection, as demonstrated by Madry et al. (2018) [22]. The third step, data protection, employs differential privacy to reduce data leakage. These initial steps establish a robust framework, ensuring AI systems are built and assessed with security in mind, which is essential for sectors such as healthcare and smart cities facing evolving cyber risks.
The fourth step, staff training, tackles automation bias, improving decision-making through transparent interfaces, as supported by Amershi et al. (2019) [38]. The fifth step, continuous monitoring, utilizes red-teaming to decrease attack success, ensuring real-time threat detection, per Ganguli et al. (2022) [25]. Research continues to refine this plan, ensuring scalability and effectiveness across all AI deployments, addressing gaps in cost and accessibility.
Table 2. 5-Step Risk mitigation Plan
Step # | Name | Description |
1 | Risk Assessment | Conducts audits to identify vulnerabilities, reducing attack surfaces. |
2 | Secure Model Design | Uses adversarial training to enhance model resilience against attacks. |
3 | Data Protection | Applies differential privacy and blockchain to secure data. |
4 | Staff Training | Addresses automation bias with training and transparent interfaces. |
5 | Continuous Monitoring | Employs red-teaming for real-time threat detection and response. |
6. Conclusion
Artificial intelligence drives innovation, yet its vulnerabilities which are model-based, data-based, and usage scenario risks, unfortunately threaten privacy, security, and trust. Model-based threats, rooted in architectural flaws, amplify risks through overparameterization, enabling attacks like prompt injection and poisoning. Data-based risks, such as leakage and inversion, stem from poor data stewardship, violating privacy and copyright laws. Usage scenario risks, including hallucinations and security vulnerabilities, emerge in deployment, with hypothetical scenarios illustrating high stakes. Many AI systems remain vulnerable, requiring comprehensive defenses to safeguard critical applications.
The Responsible AI Mitigation Framework integrates technical, governance, and human-centered strategies to address these challenges, promoting ethical deployment. Future research into scalable defenses, automated governance, quantum security, human-AI collaboration, and regulatory harmonization will ensure AI’s secure, equitable future, fostering trust and innovation.
7. Declarations
Acknowledgement
Not applicable
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Authors` Contributions
All authors have participated in drafting the manuscript. All authors contributed equally to the manuscript and read and approved the final version of the manuscript.
Conflict of Interest
The authors certify that there is no conflict of interest with any financial organization regarding the material discussed in the manuscript.
Data Availability
The data used in the article can be used for scientific purposes and shared with anyone interested in compliance with ethical principles. To share the data, it is necessary to contact the corresponding author of the article.
Ethics Approval
Ethical rules were followed within the scope of the study. No need for ethical committee approval.
Declaration of AI Usage
No generative AI tools were used for content creation in this manuscript (e.g., drafting, rewriting, or generating ideas) / AI-assisted tools were employed in this study for minor tasks such as grammar correction, language refinement, and proofreading. These tools were used transparently and in a manner that does not compromise the authors’ intellectual contribution. The authors affirm that all substantive content reflects original thought and upholds academic integrity.
References
[1] Vassilev, A., Oprea, A., Fordyce, A., & Andersen, H. (2024). Adversarial machine learning: A taxonomy and terminology of attacks and mitigations. NIST Trustworthy and Responsible AI - NIST AI 100-2e2023. https://doi.org/10.6028/NIST.AI.100-2e2023[2] Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. https://arxiv.org/abs/1412.6572v3
[3] Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., ... & Liu, Y. (2023). Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860. https://doi.org/10.48550/arXiv.2305.13860
[4] Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2020, January). How to Backdoor Federated Learning. In The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020. https://par.nsf.gov/servlets/purl/10179721
[5] Zlateva, P., Steshina, L., Petukhov, I., & Velev, D. (2024). A conceptual framework for solving ethical issues in generative artificial intelligence. In Electronics, Communications and Networks (pp. 110-119). IOS Press. https://doi.org/10.3233/FAIA231182
[6] Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P. S., Mellor, J., ... & Gabriel, I. (2022, June). Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM conference on fairness, accountability, and transparency (pp. 214-229). https://doi.org/10.1145/3531146.3533088
[7] Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, Ú., Oprea, A., & Raffel, C. (2021). Extracting training data from large language models. Proceedings of the 30th USENIX Security Symposium, 2633–2650. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting
[8] Fredrikson, M., Jha, S., & Ristenpart, T. (2015, October). Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security (pp. 1322-1333). https://doi.org/10.1145/2810103.2813677
[9] Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016). Stealing machine learning models via prediction {APIs}. In 25th USENIX security symposium (USENIX Security 16) (pp. 601-618). https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_tramer.pdf
[10] Uygun İlikhan, S., Özer, M., Tanberkan, H., & Bozkurt, V. (2024). How to mitigate the risks of deployment of artificial intelligence in medicine?. Turkish Journal of Medical Sciences, 54(3), Article 2, 483-492. https://doi.org/10.55730/1300-0144.5814
[11] Gültekin, M., & Şahin, M. (2024). The use of artificial intelligence in mental health services in Turkey: What do mental health professionals think?. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 18(1), Article 6. https://doi.org/10.5817/CP2024-1-6
[12] Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021, March). On the dangers of stochastic parrots: Can language models be too big?🦜. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623). https://doi.org/10.1145/3442188.3445922
[13] Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121-127. https://doi.org/10.1136/amiajnl-2011-000089
[14] European Commission. (2023). Proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act).
[15] National Institute of Standards and Technology. (2024). NIST Releases First 3 Finalized Post-Quantum Encryption Standards. https://www.nist.gov/news-events/news/2024/08/nist-releases-first-3-finalized-post-quantum-encryption-standards
[16] Gidney, C., & Ekerå, M. (2021). How to factor 2048 bit RSA integers in 8 hours using 20 million noisy qubits. Quantum, 5, 433. https://doi.org/10.22331/q-2021-04-15-433
[17] Sismanoglu, S., & Capan, B. S. (2025). Performance of artificial intelligence on Turkish dental specialization exam: can ChatGPT-4.0 and gemini advanced achieve comparable results to humans?. BMC Medical Education, 25(1), 214. https://doi.org/10.1186/s12909-024-06389-9
[18] Dwork, C., & Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3-4), 211-407. http://dx.doi.org/10.1561/0400000042
[19] Akgun, S., & Greenhow, C. (2022). Artificial intelligence in education: Addressing ethical challenges in K-12 settings. AI and Ethics, 2(3), 431-440. https://doi.org/10.1007/s43681-021-00096-7
[20] Huang, L. (2023). Ethics of artificial intelligence in education: Student privacy and data protection. Science Insights Education Frontiers, 16(2), 2577-2587. https://doi.org/10.15354/sief.23.re202
[21] Al Mallah, R., Halabi, T., & Farooq, B. (2023). Resilience-by-design in Adaptive Multi-agent Traffic Control Systems. ACM Transactions on Privacy and Security, 26(3), 1-27. https://doi.org/10.1145/3592799
[22] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083. https://doi.org/10.48550/arXiv.1706.06083
[23] Yuan, X., He, P., Zhu, Q., & Li, X. (2019). Adversarial examples: Attacks and defenses for deep learning. IEEE transactions on neural networks and learning systems, 30(9), 2805-2824. https://doi.org/10.1109/TNNLS.2018.2886017
[24] Klingensmith, M. (2025). Legacy Banking Costs: The Hidden Financial and Strategic Burden on Financial Institutions. FinTech Consulting. https://www.fintech-consultant.com/post/hidden-costs-of-legacy-banking-infrastructure-on-financial-institutions
[25] Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., ... & Clark, J. (2022). Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858. https://doi.org/10.48550/arXiv.2209.07858
[26] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014, January). Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014.
[27] Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018). Adversarial examples in the physical world. In Artificial intelligence safety and security (pp. 99-112). Chapman and Hall/CRC. https://doi.org/10.1201/9781351251389
[28] Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., ... & Song, D. (2018). Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1625-1634). https://openaccess.thecvf.com/content_cvpr_2018/papers/Eykholt_Robust_Physical-World_Attacks_CVPR_2018_paper.pdf
[29] Zhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023, April). Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI conference on human factors in computing systems (pp. 1-20). https://doi.org/10.1145/3544548.3581318
[30] Yang, W., Wang, S., Wu, D., Cai, T., Zhu, Y., Wei, S., ... & Li, Y. (2025). Deep learning model inversion attacks and defenses: a comprehensive survey. Artificial Intelligence Review, 58(8), 1-52. https://doi.org/10.1007/s10462-025-11248-0
[31] Gentry, C. (2009, May). Fully homomorphic encryption using ideal lattices. In Proceedings of the forty-first annual ACM symposium on Theory of computing (pp. 169-178). https://doi.org/10.1145/1536414.1536440
[32] Goldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., & Sedova, K. (2023). Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246. https://doi.org/10.48550/arXiv.2301.04246
[33] Chen, H., Zhou, H., Zhang, J., Chen, D., Zhang, W., Chen, K., ... & Yu, N. (2023). Perceptual hashing of deep convolutional neural networks for model copy detection. ACM Transactions on Multimedia Computing, Communications and Applications, 19(3), 1-20. https://doi.org/10.1145/3572777
[34] Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017, May). Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) (pp. 3-18). IEEE. https://doi.org/10.1109/SP.2017.41
[35] Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016, October). Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security (pp. 308-318). https://doi.org/10.1145/2976749.2978318
[36] Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1-38. https://doi.org/10.1145/3571730
[37] Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., Wang, L., Luu, A. T., Bi, W., Shi, F., & Shi, S. (2023). Siren’s song in the AI ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. https://doi.org/10.48550/arXiv.2309.01219
[38] Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., ... & Horvitz, E. (2019, May). Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems (pp. 1-13). https://doi.org/10.1145/3290605.3300233
[39] Gunning, D., & Aha, D. W. (2019). DARPA’s explainable artificial intelligence program. AI Magazine, 40(2), 44-58. https://doi.org/10.1609/aimag.v40i2.2850
[40] Jagielski, M., Carlini, N., Berthelot, D., Kurakin, A., & Papernot, N. (2020). High accuracy and high fidelity extraction of neural networks. In 29th USENIX security symposium (USENIX Security 20) (pp. 1345-1362). https://www.usenix.org/conference/usenixsecurity20/presentation/jagielski
[41] Yazmyradov, S., & Lee, H. J. (2024). A comprehensive review of AI security: Threats, challenges, and mitigation strategies. International Journal of Internet, Broadcasting and Communication, 16(4), 375-384. https://doi.org/10.7236/IJIBC.2024.16.4.375
[42] Voigt, P., & Von dem Bussche, A. (2017). The eu general data protection regulation (gdpr). A practical guide, 1st ed., Cham: Springer International Publishing, 10(3152676), 10-5555. https://doi.org/10.1007/978-3-319-57959-7
[43] Apruzzese, G., Laskov, P., Montes de Oca, E., Mallouli, W., Brdalo Rapa, L., Grammatopoulos, A. V., & Di Franco, F. (2023). The role of machine learning in cybersecurity. Digital Threats: Research and Practice, 4(1), 1-38. https://doi.org/10.1145/3545574
[44] OWASP. (2023). OWASP Top 10 for LLM applications. https://genai.owasp.org/llm-top-10/.
[45] Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389-399. https://doi.org/10.1038/s42256-019-0088-2
[46] Dong, T., & Huang, T. (2019). Neural cryptography based on complex-valued neural network. IEEE Transactions on Neural Networks and Learning Systems, 31(11), 4999-5004. https://doi.org/10.1109/TNNLS.2019.2955165
Author information
Authors and Affiliations
Corresponding author
Copyright information
CC-BY
