The Mechanism of Substitution
When a data subject exercises a statutory right to erasure (such as Article 17 under GDPR) against an organization operating a large language model, the standard operational response is not machine unlearning. Instead, the operator inserts the subject's identifier into a blocklist, configures system prompts to refuse generation, or applies an output classifier that suppresses matching text.
The underlying neural representation remains unchanged behind the filter. The personal data persists in the parameter weights, recoverable by adversarial prompting, jailbreaks, or membership inference attacks. *Erasure theater* names the practice after security theater: a visible, performative intervention that creates the illusion of compliance while leaving the substantive vulnerability untouched.
Output filtering is mitigation; it is not erasure. Conflating mitigation with statutory destruction leaves the gap between the legal mandate and technical reality unmeasured.
The Verification Wall
Approximate machine unlearning encounters an intractable verification problem: a model that unlearned a data point and a model that never saw the data point are indistinguishable from parameter inspection alone. A party claiming to have unlearned personal data can produce the same parameter state without having performed any unlearning.
Because the claim of weight-level forgetting is unfalsifiable from the artifact itself, compliance audits default to behavioral testing (prompting the model to see if it recites the data). Erasure theater takes advantage of this gap by placing filters at the input and output boundaries, ensuring the model fails behavioral tests even while its internal weights retain the information.
The Open-Weight Proliferation Problem
Erasure theater becomes irreversible in open-weight ecosystems. Foundation models are trained once and downloaded to hundreds of thousands of local drives, fine-tuned into derivatives, and distilled into smaller architectures. A filter applied by the original trainer reaches only the trainer's hosted API endpoint. It does not touch the parameter weights resting on decentralized infrastructure, where downstream users can remove the safety filters with a single configuration flag.
What Honest Law Requires
When technical architecture prevents verifiable unlearning, the evidentiary burden and compliance duty relocate to two points:
- Upstream Provenance: Deletion is inexpensive only before training begins. Consent screening and verifiable provenance become the operative safeguard rather than an optional preliminary check.
- Algorithmic Disgorgement: Where training incorporated personal data without verifiable rights, and where verifiable unlearning cannot be proven, destruction of the trained weights is the only non-performative remedy.
Field Notes & Ephemera
Field Note: "The company confirmed my deletion request was completed within 48 hours. Three months later, an adversarial researcher prompted the open-weight release of the base model with an inversion template and extracted my unlisted home address word-for-word. The deletion existed only in the API wrapper."