Activation-Guided GCG attacks target refusal direction in LLMs
A new arxiv paper introduces Activation-Guided GCG, an adversarial attack that optimizes suffixes by directly targeting a model's internal refusal direction in activation space, rather than output-based objectives. The work probes the geometry of safety representations, showing that refusal behavior is mediated by low-dimensional directions that can be suppressed via optimization.
Entities
Related
Penn State researchers introduce FARMA attack that poisons LLM agents' reasoning logs
Researchers at Penn State proposed FARMA, a two-phase attack that poisons an LLM agent's own decision logs and rationales rather than external knowledge sources. The attack first injects seed entries that mimic normal reasoning logs, then amplifies them to manipulate future agent behavior. This shifts the threat model for agent security beyond retrieval poisoning.
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
A new arxiv paper proposes a test-time prompt adaptation method to improve robustness of Vision-Language Models like CLIP against adversarial perturbations. The approach leverages distributional structure rather than sample-level heuristics to distinguish adversarial mispredictions from true semantic consistency.
Security researcher manually tests 10 AI attacks, reveals defenses
A security researcher published a detailed account of manually testing 10 adversarial attacks against AI systems in May 2026, including leaking a chatbot's hidden instructions and making a browsing agent perform unintended actions. The post outlines which defenses actually stopped each attack, providing practical insights for AI practitioners.
Mechanistic interpretability researchers apply causality theory to LLMs
Researchers are applying causality theory from the paper 'Causal Abstraction for Interpretability' (arXiv:2301.04709) to understand LLM internals. This approach aims to identify causal mechanisms within models, moving beyond correlation-based analysis.
SATS: Sensitivity-Aware Thresholding for MLP Activation Sparsification in LLMs
A new arxiv paper proposes Sensitivity-Aware Thresholding for Sparsity (SATS), a method to calibrate layerwise gate thresholds for MLP activation sparsification using a local sensitivity proxy instead of activation percentiles. The approach aims to reduce computation during LLM inference while preserving model quality.