0% found this document useful (0 votes)
8 views4 pages

GenAI Sec

The document outlines a checklist for Security Operations Centers (SOCs) to recognize and address threats posed by generative AI, emphasizing its dual role as a tool for adversaries and a potential attack surface. It details tactics, techniques, and procedures (TTPs) that threat actors may use, along with recommendations for updating SIEM rules and logging practices to enhance threat detection and response. The checklist is intended for various cybersecurity roles, including CISOs, threat hunters, and SOC analysts, to improve their defenses against LLM-enhanced threats.

Uploaded by

icavn1001
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
8 views4 pages

GenAI Sec

The document outlines a checklist for Security Operations Centers (SOCs) to recognize and address threats posed by generative AI, emphasizing its dual role as a tool for adversaries and a potential attack surface. It details tactics, techniques, and procedures (TTPs) that threat actors may use, along with recommendations for updating SIEM rules and logging practices to enhance threat detection and response. The checklist is intended for various cybersecurity roles, including CISOs, threat hunters, and SOC analysts, to improve their defenses against LLM-enhanced threats.

Uploaded by

icavn1001
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

🍿 LLM Enhanced Threats SOC Update Checklist

Recognize generative AI as both a tool that adversaries will weaponize and a potential attack surface in your systems.

LLM-Enhanced Threat TTPs (Microsoft’s Framework):Tactics where threat actors leverage AI to enhance traditional
attack techniques.

Who Should Use This Checklist?


★​ CISOs: Track threat actor trends and align with strategic risk decisions
★​ Cyber Threat Intelligence: Map actors / campaigns to profiles
★​ Threat Hunters: Look for behavioral patterns tied to known TTPs
★​ SOC Analysts: Prioritize threats and response
★​ Red Team / Offensive Sec: Emulate adversary TTPs during testing

General: Security Operations Center


​ Update SIEM rules to alert for both traditional and AI enhanced attacker capabilities.
​ Prioritize mitigations in high probability target areas using LLM enhanced tooling

Add SIEM Rules For

​ LLM-informed Reconnaissance
​ LLM-enhanced Scripting Techniques
​ LLM-aided Development
​ LLM-supported Social Engineering
​ LLM-assisted Vulnerability Research
​ LLM-optimized Payload Crafting
​ LLM-enhanced Anomaly Detection Evasion
​ LLM-directed Security Feature Bypass
​ LLM-advised Resource Development

SIEM: What to Log / Alert Conditions / Log Source

TTP What to Log Alert Conditions Log Sources

LLM-informed Web traffic logs Abnormal access to Web Proxies API Gateways DNS Logs
Reconnaissance (URLs, queries) API tech/vendor docs burst of Cloud SaaS Monitoring
access logs (LLM API reconnaissance-related
usage) OSINT queries unusual LLM API
scraping activities key exhaustion or traffic
spike
LLM-enhanced Script execution New script files with LLM EDR/AV LogsEndpoint Process
Scripting logsFile creation logs generation patterns Sudden LogsCloud IDE/Developer Tool Logs
Techniques (scripts)LLM API script usage from
prompts non-developer endpoints
Excessive
PowerShell/Bash/CMD
command activity

LLM-aided Source code Source code uploads with DevOps Tooling (GitHub, GitLab Audit
Development repository access AI-generated traits Logs)Sandbox/Threat Intel
Compile/build logs Low-reputation binaries FeedsSource Code Management
Malware sandbox matching GPT-style (SCM) Logs
submission logs structures

LLM-supported Email gateway Spike in multilingual Email Security Gateway (SEG)


Social Engineering logsMessaging app emailsEmails with LogsMessaging Security Logs DLP
logs (Teams, AI-optimized social Tools
Slack)Content engineering phrases
scanning for language
patterns

LLM-assisted Vulnerability scanning High-frequency vulnerability Vulnerability Scanner Logs (Qualys,


Vulnerability logsAPI usage for vuln scans Novel scan signatures Nessus)WAF LogsAPI Gateway
Research queriesWeb (non-standard nmap, burp Monitoring
application firewall scans) Prompt logs querying
(WAF) logs for CVEs

LLM-optimized Malware detection New payloads bypassing AV/EDR Logs Sandbox Analysis Email
Payload Crafting logs E-mail signatures Evasive payload Security Logs
attachment scans File delivery attempts
integrity monitoring
(FIM) logs

LLM-enhanced User behavior Synthetic login behaviors UEBA Tools (User and Entity Behavior
Anomaly Detection analytics (UBA) Unusual low and slow traffic Analytics) SIEM Native Analytics
Evasion Network traffic logs Log tampering or deletion Network Traffic Analytics (NTA)
Process injection/ attempts
evading behaviors
LLM-directed Authentication logs Failed login surges followed IAM Logs (Okta, Azure AD) Web
Security Feature (SSO, MFA events) by success CAPTCHA Application Logs Bot Detection
Bypass CAPTCHA bypass anomalies Systems
challenge/response impossible travel detections
logs Brute force
detection logs

LLM-advised Build server logs code Rapid dev cycles new C2 DevSecOps Logs Threat Intelligence
Resource repository access infrastructures or unusual Feeds Build Server Logs (Jenkins,
Development malware deployment tool uploads CircleCI)
attempts

MITRE ATT&CK / ATLAS ID and Example IOC

LLM TTP Description ATT&CK ID ATLAS ID Sample IOCs

LLM-informed Using LLMs to gather T1592, T1595 TA0031 Suspicious OSINT scraping, abnormal
reconnaissance actionable intelligence LLM API usage
on technologies &
potential vulnerabilities

LLM-enhanced Using LLMs to generate T1059 TA0002 High rate of script generation,
scripting or refine scripts to use AI-generated code artifacts
techniques in cyberattacks

LLM-aided Using LLMs in the T1587 TA0002 AI-style malware source code, fast tool
development development lifecycle of iteration
tools and programs,
including those with
malicious intent, such
as malware.

LLM-supported Leveraging LLMs for T1566 TA0003 Sophisticated phishing emails,


social engineering assistance with multilingual spear-phishing
translations &
communication, likely to
establish connections
or manipulate targets.

LLM-assisted Using LLMs to T1595.002 TA0032 Abnormal vuln search patterns,


vulnerability understand & identify AI-model queries
research potential vulnerabilities
in software & systems,
which could be targeted
for exploitation.
LLM-optimized Using LLMs to assist in T1203 TA0002 Fast-evolving obfuscated payloads
payload crafting creating & refining
payloads for
deployment in
cyberattacks.

LLM-enhanced Leveraging LLMs to T1070, T1562 TA0005 Synthetic user behavior, adversarial
anomaly detection develop methods that noise injection
evasion help malicious activities
blend in with normal
behavior or traffic to
evade detection
systems.

LLM-directed Using LLMs to find T1556, T1110 TA0035 MFA bypass attempts, CAPTCHA
security feature ways to circumvent solving patterns
bypass security features, such
as two-factor
authentication,
CAPTCHA, or other
access controls.

LLM-advised Using LLMs in tool T1587 TA0002 Rapid tool iteration, playbooks with
resource development, tool perfect grammar
development modifications, and
strategic operational
planning.

QR Link for additional Checklists


🛟 Organization AI Security Preparedness Short List
🍩 AI System Card Review Checklist
★​

🚨 2025 IR Update Checklist AI Incident Response


★​

🚨 2025 SIEM Run book for GenAI Incident Investigation


★​

🧮 AI Updates for Organization's Enterprise Risk Register


★​
★​

You might also like