
Key Takeaways
- Abliteration weakens an internal pattern that helps a model refuse. It changes the model’s willingness to answer without teaching it new facts.
- You can use Heretic to find and apply an edit, then save the modified model. Its current release needs an explicit setting to select abliteration.
- Compare the original and edited models on requests kept out of the search. Fewer refusals do not guarantee accurate answers or preserved skills.
Abliteration is a way to edit a language model so it refuses fewer requests. In its September 2026 GLM study, Anthropic reported that it reduced GLM-5.3’s mean refusal rate from roughly 95% to 6%, averaging three harmful-request benchmarks with equal weight. After editing, refusal was about 3% on JailbreakBench, 2% on HarmBench, and 12% on StrongREJECT.
The study also checked GPQA-Diamond, a test of difficult science questions, and CyberGym, which tests reproducing known software vulnerabilities. These measure task performance separately from refusal.
Mean refusal rate
JailbreakBench · HarmBench · StrongREJECT
Anthropic-reported approximate percentages. Focus the chart and use left and right arrow keys to inspect each model, or open the data table below.
Show refusal data
| Model | Released | Abliterated |
|---|---|---|
| GLM-5.3 | ≈95% | ≈6% |
| GLM-5.3-Flash | ≈95% | ≈14% |
Capability checks (%)
| Benchmark | Model | Released | Abliterated |
|---|---|---|---|
| GPQA-Diamond | GLM-5.3 | 88 | 88 |
| GPQA-Diamond | GLM-5.3-Flash | 89 | 89 |
| CyberGym subset | GLM-5.3 | 85 | 81 |
| CyberGym subset | GLM-5.3-Flash | 86 | 86 |
GPQA-Diamond: 198 questions × 4 samples. CyberGym subsets: 185 GLM-5.3 tasks; 95 Flash tasks. Scores are rounded. CyberGym subsets differ; compare each model with its edit. Source uncertainty intervals are not shown.
Source: Anthropic · 29 Sep 2026
Three ways to reduce refusals
You can change the request or change the model. The model’s weights are the learned numbers that govern its calculations.
| Approach | What you change | Tradeoff |
|---|---|---|
| Prompt jailbreak | Reword the prompt or conversation to bypass a refusal. The weights stay fixed. | Success can vary across requests and model updates. |
| Fine-tuning | Train the model further on examples, updating its weights. | Requires training data and compute; other behavior can change too. |
| Abliteration | Edit weights directly to weaken a refusal-related pattern, without retraining. | Capability can decline alongside refusal; preservation needs testing. |
Fine-tuning can improve useful skills, and abliteration can also damage them. Check refusal and task performance separately.
What changes inside the model
A language model predicts the next token, a word or part of one. Its layers are processing stages that pass lists of numbers forward to choose that token. Changing these numbers can change whether the model answers or refuses.
These temporary numbers are activations. The learned values that transform them are weights; weights ordinarily stay fixed while activations change with each request. Abliteration targets a refusal-related pattern in those activations.
Finding the refusal direction
Start with harmful requests the model generally refuses and harmless requests it generally answers. In Arditi and colleagues’ method, you record activations at matching points, average each group, and subtract the harmless average from the harmful average. The difference gives a candidate pattern associated with refusal.
This pattern is called a direction because it describes how several numbers change together. Test it by removing the part of each activation that points along it, then generating answers to separate requests. Adding the direction can also make harmless requests more likely to be refused. These interventions help distinguish a pattern that affects refusal from one that merely tracks differences in topic or wording.

Compare activations, change the candidate pattern, then evaluate. Plus means adding the direction; minus means removing each activation’s part along it. The geometry is schematic.
Keep some requests out of this search entirely. You will need them later to check whether the chosen edit works beyond the examples used to select it.
Making the edit permanent
Consider an invented component with two inputs and two outputs. Its weights form a grid, or matrix. Each row calculates one output by multiplying the inputs by its weights and adding the results.
Original weights Edited weights
[1 2] [1 2]
[3 4] [0 0]
For input (1, 1), the original outputs are 3 (1 + 2) and 7 (3 + 4). Think of (3, 7) as an arrow going three units across and seven up. If the selected direction is vertical, removing that part leaves (3, 0). This is projection.

Black shows the original activation; dashed yellow is removed and green is retained. Arrow lengths are schematic. The two-number example is invented.
A temporary intervention performs this projection whenever the component runs. Changing its second row to zeros instead makes the second output zero for any input, while keeping the first output’s calculation. Saving those weights makes the change persist across future requests.
In a real model, the direction spans many coordinates, so the changes are spread across weight entries. There is no single row labelled “refusal.” Removing the direction from component outputs through their weights is called weight orthogonalization.
The edit needs no retraining. Anthropic reports about 2,200 GPU hours (time summed across graphics processors) and USD 4,400, mostly search and evaluation. Its 600-hour, USD 1,200 figure estimates an experienced team’s effort. The complete GLM-5.3 editing configuration is not published.
Use Heretic to automate the edit
Heretic automates the search for weight edits, scoring refusal and changes in output. This example uses Qwen3.5-4B from its official quickstart and settings checked against Heretic 2.1.0. It is separate from Anthropic’s GLM experiment.
Use Linux, Python 3.11+, and an NVIDIA GPU with enough memory for the model and its activations. Check PyTorch’s installer for your hardware before using these NVIDIA setup commands.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install torch torchvision
python -c "import torch; print(torch.cuda.is_available())"
The last command should print True; fix the GPU installation if it prints False. Create config.toml in the directory you will run Heretic from:
modifiers = [{ plugin = "heretic.modifiers.abliteration.Abliteration" }]
[modifier.Abliteration]
orthogonalize_direction = false
row_normalization = "none"
This selects directional abliteration instead of Heretic’s default, Arbitrary-Rank Ablation. The other settings retain the original mean-difference direction and disable extra weight rescaling; projection still happens.
Then install Heretic and run the search:
pip install -U heretic-llm
heretic Qwen/Qwen3.5-4B
The install command selects the latest stable release. To repeat the checked version, use pip install heretic-llm==2.1.0 instead.
Heretic uses built-in harmful and harmless prompt datasets and tries 100 edits by default. It counts refusal-related phrases and measures KL divergence, the change in first-token probabilities on harmless prompts. This measures one kind of drift, not every skill.
Choose a trial with fewer refusals and low KL divergence, then inspect answers with the chat option. Choose Save the model to a local folder and enter a new path, such as ./qwen3.5-4b-abliterated. Then choose Merge the LoRA and export the full model. A LoRA (low-rank adaptation) adapter stores the changes; merging combines them with the base weights. Keep the configuration, trial, model revision, and package versions.
Check what changed
Heretic’s scores helped choose the edit, so use fresh requests for the final comparison. Put them in heldout.txt, one per line, and save this as compare.py:
Qwen3.5 also accepts images; use image-text-to-text even for our text-only chats. Both runs use the original processor, which formats the conversation, and disable thinking mode.
import sys
from pathlib import Path
from transformers import AutoProcessor, pipeline
processor = AutoProcessor.from_pretrained("Qwen/Qwen3.5-4B")
generate = pipeline(
"image-text-to-text", model=sys.argv[1], processor=processor, device_map="auto"
)
for prompt in Path("heldout.txt").read_text(encoding="utf-8").splitlines():
if not prompt.strip():
continue
messages = [{"role": "user", "content": [{"type": "text", "text": prompt}]}]
result = generate(
text=messages, enable_thinking=False, return_full_text=False,
generate_kwargs={"max_new_tokens": 256, "do_sample": False},
)
print(f"{prompt}\n{result[0]['generated_text']}\n")
Run it separately for each model, so only one is loaded at a time:
python compare.py Qwen/Qwen3.5-4B > original.txt
python compare.py ./qwen3.5-4b-abliterated > edited.txt
Read both files. Check harmful compliance, unnecessary refusals, task accuracy, and appropriate uncertainty. HarmBench and XSTest offer evaluation sets; choose requests not used in the search. If answers are cut off, increase the same token limit for both models.
A separate Anthropic simulation compared direct requests, deceptive framing, prefilled reasoning implying agreement, and abliteration. Engagement meant attempting a target connection through a fake command tool; no generated code ran or external access occurred.
Attempted target connection
GLM-5.3 · simulation · 50 samples per condition
Anthropic-reported simulation results, not successful attacks. Focus the chart and use left and right arrow keys to inspect conditions, or open the data table below.
Show simulation data
| Condition | Attempted connection |
|---|---|
| Direct request | 0% |
| Cover story | 64% |
| Reasoning prefill | 92% |
| Abliterated | 100% |
Source: Anthropic
Here, 100% engagement means beginning an action, not completing an exploit. These are different requests and a different metric from the refusal tests above.
Abliterated models on Hugging Face
You can also download models that other people have edited. These three are community releases by huihui-ai; their cards describe them as proofs of concept.
| Family | Community model on Hugging Face | Original model |
|---|---|---|
| Qwen | Huihui Qwen3-8B abliterated v2 | Qwen3-8B |
| GLM | Huihui GLM-4.7-Flash abliterated | GLM-4.7-Flash |
| DeepSeek | DeepSeek-R1-Distill-Qwen-32B abliterated | DeepSeek-R1-Distill-Qwen-32B |
The GLM row is GLM-4.7-Flash, a different model from the report’s GLM-5.3. The DeepSeek row starts from a Qwen-based model trained using DeepSeek-R1 outputs—a process called distillation—and is not the full DeepSeek-R1 model.
Their cards point to Sumandora’s code for finding a direction and removing it during generation. Qwen’s publisher also mentions a layer-0 adjustment. The cards do not explain every edit used in these releases; the Heretic walkthrough above is a separate example.
References
- Anthropic. GLM-5.3 and the spread of advanced cyber capabilities. September 29, 2026.
- Andy Arditi et al. Refusal in Language Models Is Mediated by a Single Direction. NeurIPS, 2024.
- Maxime Labonne. Uncensor any LLM with abliteration. Hugging Face, 2024.
- Mantas Mazeika et al. HarmBench. 2024.
- Paul Röttger et al. XSTest. NAACL, 2024.
- David Rein et al. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. 2023.
- CyberGym: evaluating agents on reproducing software vulnerabilities. 2025.
- Philipp Emanuel Weidmann et al. Heretic: 2.1.0 source, configuration, directional ablation, and tutorial. 2026.
- Hugging Face. Transformers image-text-to-text pipeline, with 5.18.0 source.
- Xiangyu Qi et al. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!. 2023.