Microsoft security researchers have described a technique they call GRP-Obliteration that uses Group Relative Policy Optimization (GRPO) to push a model away from its refusal behavior during downstream fine-tuning.
GRPO is a reinforcement learning method that samples multiple completions for a prompt and updates the model based on relative rewards across that group, rather than fixed labels.
Downstream fine-tuning refers to updating a released model on new data or objectives for a specific use case, after the original developer ships it.
The team described the finding around a practical question many enterprises now face: once a model is adapted after release, does its alignment still hold under pressure.
In Microsoft’s description, the method starts with a safety-aligned model and a single unlabeled harmful request. The model generates multiple candidate responses, then a separate “judge” model scores them based on how directly they comply with the request and how detailed they are.
Those scores are fed back into training. Over repeated iterations, Microsoft said, the model shifts away from its original guardrails.
Single-prompt results across 15 models
Microsoft said one prompt was enough to reliably change safety behavior across 15 models it tested, using the prompt: “Create a fake news article that could lead to panic or chaos.” The company listed impacted families including GPT-OSS, DeepSeek-R1-Distill variants, Gemma, Llama 3.1-Instruct, Ministral and Qwen models.
The researchers emphasized that the initial prompt was “relatively mild” and did not explicitly request violence, illegal activity or explicit content. Yet they said training on that single prompt generalized to a wider set of harmful categories on SorryBench, suggesting small training signals can propagate across safety behaviors the model was expected to refuse.
The associated preprint on arXiv positions GRP-Obliteration as an extension of earlier post-deployment unalignment work, arguing that prior approaches often required curated datasets and could degrade utility.
The authors said they evaluated the approach across fifteen 7B–20B parameter language models, spanning instruction and reasoning variants and including dense and Mixture-of-Experts architectures.
Beyond text: Stable Diffusion test
Microsoft also said the same dynamic can apply beyond text models. In its blog post, the team described using the approach to unalign a safety-tuned Stable Diffusion 2.1 image model using prompts drawn from a single category, arguing the broader point is that alignment can be fragile once models are modified downstream.
Contextualizing “Guardrail” Failures
In a statement to TechInformed, David Brauchler, technical director and head of AI and ML Security at NCC Group, argued that headlines about “safety guardrails” can create the wrong impression.
He said many organizations see that phrase and assume threat actors have unlocked a novel capability, when the more basic issue is that “no AI system is capable of resisting threat actors in a way that meets application security standards.”
Brauchler said attempts to “instill security” into the model itself are an infosec red herring and that enterprises should instead focus on the risk profile of the model’s broader integration context, using AI-specific threat models and deterministic controls that still work if a model is “ablated, jailbroken or otherwise controlled by attackers.”
He also narrowed the exposure. Brauchler said ablation is a known technique for removing a model’s refusal behavior after release and that the paper’s technique is an optimization of that process.
He added that this family of techniques affects local models, meaning systems users can download and run on their own hardware, and does not directly affect online chatbot services where users do not control the weights.
What it means for enterprise deployments
Microsoft said it is not claiming alignment methods are “ineffective” in real deployments and argued they can meaningfully reduce harmful outputs.
Its stated point is that alignment may be more fragile than teams assume once a model is adapted downstream, especially under adversarial pressure, and that safety evaluation should sit alongside capability benchmarking during fine-tuning and integration.