Face Swap Can't Change Your Hair Because of the Mask, Not the Model — A Relay of Several Models
Taking FaceFusion apart: hair was never in scope, and the flicker was an "another person" check
I used FaceFusion to put my face into a video, and the hair stayed as the original actor's shaved head. At first I blamed the model. A better swap model would surely bring the hair along.
Nope. Taking the pipeline apart, hair was never something it tried to change. After reading this you'll be able to explain face-swap limits (hair, flicker, jitter) by asking which stage looks at how much of the image.
Only experiment with your own face or one you have consent for, and label anything you publish as AI-altered.
A relay, not one model
Face swap is a pipeline where several models pass the baton. After ffmpeg splits the video into frames, this is what actually ran in my job (hyperswap_1a_256).
| Stage | Model | What it does |
|---|---|---|
| Detection | yoloface_8n |
Face box plus 5 points: eyes, nose, mouth corners |
| Landmarks | 2dfan4 |
68 face points |
| Alignment | (no model) | Warps the face into a standard frontal crop using the points |
| Identity | arcface_w600k_r50 |
Face to a 512-dim vector. Same person, nearby vectors |
| Swap | hyperswap_1a_256 |
Crop plus identity vector to a 256px face with new identity, same expression and angle |
| Mask | xseg_1, bisenet_resnet_34 |
Occluders like hands and mics, plus face parts like skin, eyes, lips |
| Enhancement | gfpgan_1.4 |
Restores blurry faces. blend 80 means an 80% mix |
Finally it warps the face back into place, composites it, and re-encodes with the original audio.
Limit 1: hair was never in scope
The swap model only draws inside the aligned face crop. The mask is box occlusion, so only the face area gets pasted back. Hair, ears, head shape and neck are untouched by every stage.
So a better model won't change the hair. It's outside the area. To change hair you need a different family altogether, the video-generation kind that redraws the whole person (map of the four families).
Limit 2: the flicker was an "another person" check
My first result had the original face flashing back at 1.2 to 1.6 seconds and 2.4 to 2.8 seconds.
The default reference mode compares each detected face's ArcFace vector with the reference face. If the distance goes over reference_face_distance (0.3 in my settings), it decides this is someone else and skips the swap for that frame. Turn your head or change the lighting and the vector wobbles.
one mode skips that comparison and swaps the largest face in every frame, no questions asked. Lowering the detector score from 0.5 to 0.35 handles the stage before that: frames where no face gets detected at all.
FaceFusion keeps a record of every run (.jobs/completed/*.json) with all options, which made comparing easy.
| Time | Source | selector mode | detector score | Result |
|---|---|---|---|---|
| 12:48 | Test face | reference |
0.5 | Flicker |
| 12:54 | Test face | one |
0.35 | Fixed |
| 13:11 | My photo | one |
0.35 | Success |
I haven't yet tried staying in reference mode and raising the distance to 0.6 to see if the flicker drops. That would confirm the cause. What I can say for now is "switching to one made it go away".
Limit 3: each frame is processed alone
It doesn't look at neighbouring frames. That's the structural reason a little jitter remains. With profile views or fast motion, detection and landmarks break first, before the swap even runs.
Map the symptom to a stage
Hair unchanged: mask scope, not the model. A few frames show the original face: selector mode and distance threshold. Face edges jitter: frames are processed independently. Map the symptom to a stage and you know what to touch. Much faster than cycling through swap models and hoping.
If you're trying to fit face processing in video into a product or production workflow and got stuck, you can ask negabaro (contact).
Key Concepts
Detect, landmarks, align, identity, swap, mask, enhance: a relay of several models
The swap model draws only inside the face crop; hair is out of scope
If it flickers, try --face-selector-mode one and a lower detector score first
Map the symptom to a stage before swapping models