Research Reference

Demonstrates that suffix-based adversarial attacks can reliably jailbreak aligned LLMs, challenging the robustness of RLHF-based safety training.

The definitive practitioner reference for LLM security risks — prompt injection, insecure output handling, supply chain vulnerabilities, and more.

First systematic analysis of prompt injection in real-world LLM integrations, introducing the concept of indirect injection via external data sources.

Evaluates detection and mitigation strategies against adversarial prompt attacks including perplexity filtering and paraphrasing.

Practical defense architectures for LLM deployments, covering sandboxing, output validation, and privilege separation patterns.

Anthropic's framework for systematic adversarial evaluation of LLMs, covering threat taxonomies and structured attack methodologies.

NIST guidance for assessing and managing risks specific to AI systems across the full development and deployment lifecycle.

Proceedings covering adversarial robustness, privacy attacks on ML models, and emerging AI security research from the 2024 IEEE SaTML conference.

Security risks specific to autonomous AI agents — unsafe tool invocation, goal drift, multi-agent trust exploitation, and data exfiltration patterns.

Foundational paper introducing the ReAct pattern that enables LLMs to interleave reasoning and action, forming the basis of modern agentic architectures.

ToolEmu: automated evaluation of LM agent risks across tool use scenarios, revealing systematic failure modes in production-like environments.

Comprehensive benchmark covering 10 agent attack types across 55 test cases, providing a structured framework for measuring agentic AI security posture.

Demonstrates indirect prompt injection attacks against computer-use agents via manipulated screen content, highlighting a new attack surface.

Analysis of goal misalignment and catastrophic risk in fully autonomous AI systems, with recommendations for oversight mechanisms.

Security analysis of the MCP standard for AI tool integration, covering trust boundaries, authorization risks, and recommended mitigations.

Research on authentication and trust propagation in multi-agent pipelines, covering orchestrator compromise and inter-agent injection vectors.

The foundational paper describing the ATT&CK framework's design, intended use for adversary behavior modeling and detection gap analysis.

Original Sigma specification paper describing the YAML-based signature format for SIEM-agnostic detection rule authoring.

Kyle Bailey's framework for measuring and advancing detection engineering capability maturity across people, process, and technology dimensions.

Palantir's ADS framework for structuring detection hypotheses, covering goal, categorization, technical context, and response guidance.

David Bianco's classic model illustrating the relative difficulty of denying adversaries different types of IOCs — from hash values to TTPs.

Engineering patterns for implementing distributed behavioral detection pipelines using Apache Spark — handling class imbalance, windowing, and UDFs.

Foundational adversarial ML paper on evasion attacks — essential for understanding how detection ML models can be evaded.

Analysis of production detection rule patterns across enterprise deployments, covering rule quality metrics and coverage distribution.

CERT/CC's authoritative guide to insider threat program development, incident analysis, and technical controls for detection.

CERT research defining a structured ontology for insider threat indicators across technical and behavioral signal categories.

Statistical methods for building behavioral baselines, detecting drift, and managing false positive rates in enterprise UEBA deployments.

Comprehensive survey of insider threat detection techniques including anomaly detection, machine learning, and psychological indicators.

Using graph analytics on enterprise activity data to detect lateral movement and privilege abuse patterns indicative of insider threats.

Reference documentation for the widely-used CERT synthetic insider threat dataset, covering log types, scenario descriptions, and evaluation methodology.

Techniques for training anomaly detectors under extreme positive-class scarcity — oversampling, cost-sensitive learning, and evaluation framework.

First systematic study of prompt injection in deployed GPT-3 applications, introducing direct and indirect injection taxonomy.

Analysis of failure modes in RLHF safety training, identifying competing objectives and generalization failures that enable jailbreaks.

Survey of training-time backdoor attacks against LLMs — poisoning datasets to create triggered behaviors that bypass safety evaluations.

How LLMs memorize and leak training data through inference-time attacks — membership inference, extraction attacks, and mitigations.

Comprehensive taxonomy of LLM jailbreak techniques in the wild, with analysis of defense effectiveness across attack categories.

Demonstrates that LLMs memorize and reproduce verbatim training sequences, enabling PII and sensitive data extraction at scale.

Randomized smoothing defense against adversarial prompt attacks — the first provably robust defense for aligned LLMs.

Practical threat modeling guide for LLM-integrated applications, covering attack surfaces, trust boundaries, and monitoring recommendations.