Generative AI Security Risks Overview
Uploaded by
bsrpropGenerative AI Security Risks Overview
Uploaded by
bsrpropCommon questions
Powered by AICross-tenant data leakage occurs when one customer's data becomes accessible to another due to model/service misconfiguration, leading to massive data breaches and reputational damage. Effective measures to prevent it include tenant isolation, enforcing access boundaries, and encrypting data by tenant ID .
Prompt injection risk occurs when malicious users manipulate prompts to alter the behavior of Language Learning Models (LLMs) or bypass restrictions, leading to unauthorized access, misinformation, or policy violations. It can be mitigated through input sanitization, prompt templating, and user prompt isolation .
Inference-time privacy breach risk involves sensitive user inputs being logged, reused, or exposed in prompt history, potentially leading to data breaches and compliance violations. Mitigation strategies include using ephemeral prompt storage, encryption, and providing opt-out mechanisms to protect user privacy .
Unauthorized access to AI systems occurs due to weak access controls, allowing attackers or insiders to misuse GenAI interfaces, potentially exposing protected APIs and leading to an escalation of privileges. Security can be enhanced by implementing role-based access control (RBAC), multi-factor authentication (MFA), audit logging, and session timeouts .
Model memorization of sensitive data results in regulatory violations (e.g., GDPR) and reputational harm because LLMs may recall and leak sensitive training data. This can be mitigated by minimizing data during training, ensuring data de-identification, and conducting training data audits to maintain compliance and protect privacy .
Unintentional generation of harmful or biased content by AI can result in brand damage, legal exposure, and regulatory scrutiny. To mitigate these risks, organizations should use output moderation tools, include human-in-the-loop reviews, and conduct fairness testing to ensure more ethical and safe AI outputs .
Consent and transparency are crucial to comply with legal requirements (e.g., GDPR Article 13) and maintain user trust, as users need to be aware of how their inputs are used. Organizations can ensure compliance by providing clear disclosures, tracking consent, and offering opt-in mechanisms for data reuse .
Training on proprietary or sensitive data can lead to the reuse or leakage of trade secrets, violating contracts. Companies can manage these risks by classifying data before training, utilizing synthetic data, and enforcing strict training data governance to protect sensitive information .
Third-party models or plugins introduce risks such as data exposure, plugin hijacking, and dependency attacks. Organizations can mitigate these risks by employing plugin sandboxing, conducting supplier security assessments, and using allowlisted extensions to ensure secure integration of third-party tools .
Data leakage via model output can expose personally identifiable information (PII), confidential company data, or intellectual property. This exposure may occur because LLMs can unintentionally output sensitive inputs from their training data. To prevent this, strategies such as implementing differential privacy, output filtering, and data redaction before inference are recommended .