The Domain Is a Residue:

Adapting Self-Supervised Features, Not Generators

Thomas Deixelberger & Markus Steinberger
Huawei Technologies, Austria
Graz University of Technology, Austria
arXiv Code (coming soon)

How it works

Only the Representation Feature Adapter (RFA) is trained. The DINO encoder and the Sana decoder stay frozen.

Input: a horse on grass.
DINO
The DINO feature map of the horse, shown as colours.
RFA
The feature map after the adapter: the same scene.
Sana
Output: the same animal as a zebra.
Inputhorse
Encoderfrozen
Feature mapscene + a small residue
AdapterRFA, trained
Adapted mapsame scene, residue moved
Decoderfrozen
Outputzebra
Where the adapter changes the feature map, drawn over the input: red where it acts, yellow where it acts most. The grass stays untouched.
where it actsred = changed, yellow = changed most
TL;DR

In self-supervised features, fog, night and the look of a game engine are a residue, not the scene. We move that residue with a small adapter, never touching the generator, and still match or beat generators retrained from scratch.

Remove the weather

top: real adverse input, bottom: RFA output

Fog
Input: a road in dense fog.
RFA output: the same road, fog removed.RFA
Snow
Input: a snowy street crossing.
RFA output: the same crossing, snow removed.RFA
Rain
Input: a wet street at a traffic light in rain.
RFA output: the same street, dry.RFA
Haze
Input: a hazy river with hidden mountains.
RFA output: the river clear, mountains visible.RFA
Night
Input: a downtown street at night.
RFA output: the same street by day.RFA

Add the weather

one clear street, four RFA outputs

Clear input
Input: a clear, sunny street.
+ Fog
The same street with fog added by the adapter.RFA
+ Snow
The same street with snow added by the adapter.RFA
+ Rain
The same street with rain added by the adapter.RFA
+ Night
The same street with night added by the adapter.RFA

Sim to real

top: game render, bottom: RFA output

Sim → Real
Input: game render 1.
RFA output: game render 1 as a photograph.RFA
Sim → Real
Input: game render 2.
RFA output: game render 2 as a photograph.RFA
Sim → Real
Input: game render 3.
RFA output: game render 3 as a photograph.RFA
Sim → Real
Input: game render 4.
RFA output: game render 4 as a photograph.RFA
Sim → Real
Input: game render 5.
RFA output: game render 5 as a photograph.RFA

BibTeX

@misc{deixelberger2026residue,
  title         = {The Domain Is a Residue: Adapting Self-Supervised Features, Not Generators},
  author        = {Thomas Deixelberger and Markus Steinberger},
  year          = {2026},
  eprint        = {2609.37330},
  archivePrefix = {arXiv}
}