The Problem of Selective Forgetting
Training an artificial neural network is a non-linear, irreversible optimization process. Gradient descent adjusts billions of floating-point parameters simultaneously based on the full training corpus. Information does not reside in a localized address or modular slot; it is distributed across the weight matrix. When personal or sovereign data is ingested, it is metabolized into the model's global representations.
Standard data protection laws (such as GDPR Article 17) grant data subjects a right to erasure without undue delay (typically interpreted as 30 days). For frontier models costing hundreds of millions of dollars and months of continuous compute to train, retraining from scratch per deletion request is economically impossible. Machine unlearning was conceived to bridge this gap.
Architectures of Deletion
Over a decade of research, computer science developed three primary approaches to unlearning:
- Exact Retraining via Partitioning (SISA): Sharded, Isolated, Sliced, and Aggregated (SISA) training divides the dataset into disjoint shards and trains independent sub-models. When a deletion request arrives, only the affected shard is retrained. While exact, this architecture must be adopted prior to initial training and introduces substantial memory and latency overheads.
- Certified Removal: Formulates differential-privacy-style mathematical bounds guaranteeing that the distribution of model weights after removal is statistically indistinguishable from a model trained without the deleted point. This approach is computationally bounded and applies primarily to convex or linear model classes, remaining intractable for large-scale deep transformer architectures.
- Approximate Unlearning (Gradient Ascent & Weight Surgery): Interventions that fine-tune weights using negative gradients on the target data, or ablate specific attention heads and MLP neurons. This is the cheapest approach for large models, but it degrades general capabilities and leaves residual latent representations intact.
The Verification Wall
Approximate unlearning encounters a fundamental epistemological limit: a model that unlearned a data point and a model that never saw that data point can produce identical parameter states. A party claiming to have executed machine unlearning can present weights without having performed any real removal, because forgetting is unfalsifiable from the model artifact itself.
Exact erasure is possible in principle and ruinous in practice, while approximate erasure is practical in every respect except the one that matters: no data subject, auditor, or court can verify that it occurred.
Intersection with the Baked-In Paradox
Because reliable, verifiable unlearning at foundation scale does not exist, claims of compliance default to the Baked-In Paradox: once semantic capital is absorbed into neural parameters, the contamination is structural and permanent. When companies use input/output filters to mimic unlearning without altering weights, the practice constitutes Erasure Theater, leaving Weight Incarceration active.
Field Notes & Ephemera
Field Note: "The operator claimed the biographical record had been 'unlearned' via weight projection. When we probed the latent representations with orthogonal steering vectors, the model completed the private phone number with 98% token confidence. The parameter had not forgotten; it had merely been silenced."