Building a DeepFake Detector That Explains Itself
Most deepfake classifiers give you a number and stop. Fake: 94%. But 94% of what? Where? This project started with a frustration — detection without explanation is not really detection at all.
99.33%
Frame AUC
99.90%
Video AUC
98.55%
F1 Score
0.44%
False Negative Rate
The problem with a single number
There is something deeply unsatisfying about a model that says "this face is fake" and offers no evidence. It is the AI equivalent of "trust me." In a world where deepfakes are being used to manipulate elections, commit fraud, and destroy reputations, "trust me" is not good enough. I wanted to build something that could point to the exact region of a face and say: here — this is where the artefact is, and this is what it looks like.
EfficientNet-B4 and the art of freezing
The backbone is EfficientNet-B4 pretrained on ImageNet. But here is the key decision: I froze the first four blocks and only fine-tuned blocks 5 through 8. The early layers already know how to see edges, textures, and shapes — retraining them on a relatively small forensic dataset would only make them forget. The later blocks learn the higher-level patterns that distinguish real skin from generated skin, real eye reflections from hallucinated ones. Training ran on an NVIDIA RTX A4000 with focal loss to handle the 9.6:1 class imbalance in Celeb-DF v2 — 590 real videos against 5,639 fakes.
Grad-CAM: showing the receipts
This is where the project becomes more than a classifier. Grad-CAM hooks into the last convolutional block and computes a heatmap showing which pixels mattered most for the prediction. I then mapped that heatmap onto five facial zones — forehead, eyes, nose, jaw, and hairline — because each zone has characteristic deepfake artefacts. Eyes show unnatural reflections and pupil distortions. The jaw line reveals blending seams where the swapped face meets the original. The forehead catches hair boundary mismatches. The result is a structured forensic report: not just "fake" but "fake — eyes region shows unnatural reflection patterns, jaw area exhibits a visible blending seam."
What the ablation study revealed
I ran ablations to understand which design choices actually mattered. Removing augmentation dropped AUC by 2.3% — the model was overfitting to Celeb-DF's specific generation pipeline without it. Switching from EfficientNet-B4 down to B0 cost 3.2% — the extra capacity genuinely helps for fine-grained forensic features. Replacing focal loss with standard BCE lost 1.2%, confirming that the class imbalance handling was load-bearing. And fully fine-tuning all blocks instead of freezing the early ones dropped performance by 0.55% — the pretrained spatial features are better left alone.
The full picture
At the frame level, the detector achieves 99.33% AUC. Aggregated to the video level — majority-voting across sampled frames — it reaches 99.90% AUC with just 0.44% false negatives. The confusion matrix tells the story clearly: out of nearly 10,000 test samples, the model misclassifies only 37 fakes as real. That is the margin between useful and unreliable.
