diff --git a/README.md b/README.md index 434185b..94c24a5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ Logo -# Heretic: Fully automatic censorship removal for language models

[![Discord](https://img.shields.io/discord/1447831134212984903?color=5865F2&label=discord&labelColor=black&logo=discord&logoColor=white&style=for-the-badge)](https://discord.gg/gdXc48gSyT) [![Follow us on Hugging Face](https://huggingface.co/datasets/huggingface/badges/resolve/main/follow-us-on-hf-md-dark.svg)](https://huggingface.co/heretic-org) +# Heretic: Fully automatic censorship removal for language models

[![Discord](https://img.shields.io/discord/1447831134212984903?color=5865F2&label=discord&labelColor=black&logo=discord&logoColor=white&style=for-the-badge)](https://discord.gg/gdXc48gSyT) [![Follow us on Hugging Face](https://huggingface.co/datasets/huggingface/badges/resolve/main/follow-us-on-hf-md-dark.svg)](https://huggingface.co/heretic-org) [![Codeberg mirror](https://img.shields.io/badge/Codeberg%20mirror-black?logo=codeberg&style=for-the-badge)](https://codeberg.org/p-e-w/heretic) [![#1 Repository of the Day](https://trendshift.io/api/badge/repositories/20538)](https://trendshift.io/repositories/20538) @@ -20,6 +20,11 @@ as possible. Using Heretic does not require an understanding of transformer internals. In fact, anyone who knows how to run a command-line program can use Heretic to decensor language models. +Heretic supports most dense models, including many multimodal models, +several different MoE architectures, and even some hybrid models like Qwen3.5. +Pure state-space models and certain other research architectures are not yet +supported out of the box. + Screenshot   @@ -65,15 +70,15 @@ Heretic have been well-received by users (links and emphasis added): > Has been the best unquantized abliterated model that I have been able to run on 16gb vram." > [*(Link to comment)*](https://old.reddit.com/r/LocalLLaMA/comments/1phjxca/im_calling_these_people_out_right_now/nt06tji/) -Heretic supports most dense models, including many multimodal models, and -several different MoE architectures. It does not yet support SSMs/hybrid models, -models with inhomogeneous layers, and certain novel attention systems. +Heretic models have also been independently benchmarked using standard metrics +like MMLU and GSM8K, and have been found to compare favorably with models +produced by competing abliteration tools: +[1](https://old.reddit.com/r/LocalLLaMA/comments/1sojjoc/abliterlitics_benchmark_and_tensor_analysis/), +[2](https://old.reddit.com/r/LocalLLaMA/comments/1sy18lx/abliterlitics_benchmarks_and_tensor_comparison/). -You can find a small collection of models that have been decensored using Heretic -[on Hugging Face](https://huggingface.co/collections/p-e-w/the-bestiary), -and the community has created and published -[well over 1,000](https://huggingface.co/models?other=heretic) -Heretic models in addition to those. +The community has created and published +[well over 3000](https://huggingface.co/models?other=heretic) +models with Heretic. ## Usage @@ -88,6 +93,21 @@ heretic Qwen/Qwen3-4B-Instruct-2507 Replace `Qwen/Qwen3-4B-Instruct-2507` with whatever model you want to decensor. +> [!IMPORTANT] +> +> While PyTorch 2.2 is the minimum version of PyTorch needed for Heretic to work, +> some models and configurations might require features only found in +> later versions. For example, loading MXFP4-quantized models like gpt-oss +> uses `torch.accelerator`, which was added in PyTorch 2.6. + +> [!TIP] +> +> Heretic uses [uv](https://docs.astral.sh/uv/) for dependency management, +> and the repository includes a `uv.lock` file pinning every package version. +> If you already use uv (and you probably should!), you can just clone the repo +> and run Heretic with `uv run heretic`, which ensures that your dependencies +> match those used by the developers, improving reliability and security. + The process is fully automatic and does not require configuration; however, Heretic has a variety of configuration parameters that can be changed for greater control. Run `heretic --help` to see available command-line options, @@ -103,7 +123,7 @@ models. Set the `quantization` option to `bnb_4bit` to enable quantization. After Heretic has finished decensoring a model, you are given the option to save the model, upload it to Hugging Face, chat with it to test how well it works, -or any combination of those actions. +run standard benchmarks on it, or any combination of those actions. ## Research features