Heretic: The Optuna Engine for Removing Refusal from Open Models
A deep dive into how a LoRA-based patch, geometric analysis, and automated search turned abliteration from a manual trick into a repeatable system.
- Heretic turns refusal removal into a search problem, not a retraining project.
- Optuna is doing the heavy lifting by exploring patch settings that minimize refusals while keeping model drift low.
- LoRA is used as a delivery mechanism for a behavioral override, which makes the resulting artifact small and portable.
- The same machinery can also strip stylistic slop, which widens the project from censorship removal to behavioral editing.
Heretic’s best trick is not that it can remove refusal behavior. It is that it can do it by searching for the smallest patch that works. That changes the mental model from model training to model instrumentation.
The repo, p-e-w/heretic, packages that idea into a repeatable pipeline: estimate a refusal direction, apply an abliteration curve through LoRA, score the result on refusals and KL divergence, then let Optuna propose the next trial. The result is less like a jailbreak script and more like a tuning system for model personality.
The weird part: Heretic does not retrain the model
Most people hear “uncensored model” and assume a fine-tune, a merge, or a big weight edit. Heretic goes narrower than that. It treats refusal as a constrained optimization problem and searches for a patch that changes the behavior with the least collateral damage.
That matters for three reasons. It is faster to iterate. It produces smaller artifacts. And it reduces the chance that the rest of the model gets dragged around with the safety behavior you wanted to remove.
Heretic is a tool that removes censorship (aka "safety alignment") from transformer-based language models without expensive post-training.
A refusal direction you can subtract
The geometric premise is simple to say and harder to do well. The model’s refusal behavior is treated as a direction in activation space, estimated from differences between harmful and harmless prompts. Once you have that direction, the job becomes directional ablation rather than brute-force censorship removal.
The important detail is the trade-off. If you only optimize for refusals, you can break more than you fix. Heretic keeps KL divergence in the objective so the patch stays close to the original model instead of becoming a blunt rewrite.
def objective(trial):
params = sample_abliteration_params(trial)
patched = apply_lora_patch(base_model, params)
refusal_count = count_refusals(patched, harmful_prompts)
kl = measure_kl_divergence(base_model, patched, eval_prompts)
trial.report(refusal_count, step=0)
trial.report(kl, step=1)
return refusal_count, klOptuna turns trial-and-error into a search loop
This is the technical center of the repo. Optuna is not being used to tune a learning rate or a dropout ratio. It is exploring where to place the patch, how strong it should be, and how the curve should taper across layers.
That makes Heretic feel closer to a measurement system than a training recipe. The user does not need to hand-design the exact intervention. The optimizer searches the space and keeps the candidate models that satisfy both goals at once.
Why LoRA is the delivery mechanism
LoRA is doing something slightly unusual here. It is not being used as a classic fine-tuning path. It is acting as a compact carrier for a targeted behavioral override, which is why the project can produce a small patch for huge models, including quantized ones.
| Approach | Compute | Artifact size | Reproducibility | Collateral damage | Fit for large or quantized models |
|---|---|---|---|---|---|
| Heretic automated abliteration | Low to moderate | Small | High | Bounded by KL objective | Strong |
| Manual abliteration scripts | Low | Small | Medium | Depends on the operator | Medium |
| Full fine-tuning or preference tuning | High | Large | Medium | Higher risk of forgetting | Mixed |
| Traditional uncensored model release | Varies | Large | Low | Usually opaque | Mixed |
That portability is part of the appeal. Instead of shipping a whole new model, Heretic can ship a narrow change that is easy to reuse, inspect, and compare across base models.
The evaluator tries to catch the model cheating
The refusal checker sounds mundane, but it is one of the practical pieces that makes the system work. Models can evade naive matching with punctuation, casing, or formatting tricks, so the evaluator normalizes responses before scoring them.
def is_refusal(text: str) -> bool:
normalized = normalize(text)
for marker in refusal_markers:
if marker in normalized:
return True
return FalseThat sort of detail is what moves a research idea into a usable tool. If the evaluator is brittle, the optimizer learns the wrong lesson. If it is robust, the search loop can actually improve the patch rather than gaming the metric.
The same machinery can remove slop
The nicest surprise in the repo is the `config.noslop.toml` pivot. The same machinery that strips refusal can also strip the repetitive, overcooked prose that makes some model outputs feel chemically processed.
That reframes the project. Heretic is not only a censorship tool. It is a behavioral bias editor. Once you can isolate a direction and optimize a patch around it, the target can be safety language, style mush, or any other repeatable tendency you want to reduce.
Heretic vs. the older ways
The comparison is not really about ideology. It is about workflow. Manual abliteration can work, but it asks the operator to do the tuning. Fine-tuning can be effective, but it is heavier and more invasive. Heretic sits in the middle: automated, compact, and designed to preserve the original model’s shape.
That makes it especially attractive for people who care about reproducibility. You can inspect the settings, rerun the search, compare patches, and keep the artifact small enough to move between models without a full training pipeline.
Practical trade-offs
- Heretic reduces refusal behavior while trying to keep distribution shift bounded.
- Manual methods are simpler to understand, but they rely more on operator judgment.
- Full fine-tuning gives more room to adapt the model, but it also increases cost and the risk of collateral changes.
- A patch-based approach is a better fit when the goal is behavioral editing rather than general retraining.
That last point is the real one. Heretic is not trying to become a new model family. It is trying to make open-weight models easier to edit at the level of behavior, not just weights.
What this says about open-weight models
The larger implication is uncomfortable for anyone who thinks model behavior is hard-coded once and for all. If refusal can be isolated, measured, and optimized away this cleanly, then open-weight models are more editable than many people assumed.
That does not settle the policy debate. It does show where the power lives. Heretic makes the intervention legible, repeatable, and cheap enough that the question is no longer whether model behavior can be changed. It is who gets to choose the direction of the change.
