unsupervised domain adaptation for semantic segmenta- tion ... · [3] tsai et al., learning to...

The Problem of Domain Gap

Unsupervised Domain Adaptation

Proposed Iterative Framework

Proposed iterative self-training framework. Left: algorithm work-flow. Right: results on Cityscapes before and after adaptation.

Experiment: GTA5 —> Cityscapes

Experiment: SYNTHIA —> Cityscapes .

Exp: Cityscapes —> NTHU [2]

Reference [1] Wu et al., Wider or deeper: Revisiting the resnet model for visual recognition, arXiv 2016. (ResNet-38) [2] Chen et al., No more discrimination: Cross city adaptation of road scene seg-menters, ICCV 2017. (GCAA) [3] Tsai et al., Learning to adapt structured output space for semantic segmentation, CVPR 2018. (MAA) [4] Zhang et al., Curriculum domain adaptation for segmentation of urban scenes, ICCV 2017. (Curr. DA) [5] Hoffman et al., FCNs in wild: Pixel-level adversarial and constrained adaptation, arXiv 2017. (FCNWild) [6] CyCADA: Cycle-consistent adversarial domain adaptation, ICML 2018. (CyCADA) [7] Saito et al., Adversarial dropout regularization, ICLR 2018. (ADR) [8] Murez et al., Image to Image Translation for Domain Adaptation. (I2I Adapt)

Unsupervised Domain Adaptation for Semantic Segmenta-tion via Class-Balanced Self-Training

Yang Zou*, Zhiding Yu*, B. V. K. Vijaya Kumar, Jinsong Wang yzou2@andrew.cmu.edu, zhidingy@nvidia.com, kumar@ece.cmu.edu, jinsong.wang@gm.com

Preliminaries and Definitions

Self-Training with Self-Paced Learning

SPL Policy Design and Spatial Priors

Cityscapes —> Cityscapes GTA5 —> Cityscapes Fine-tuning for Supervised Domain Adaptation

where:

Self-Training for Unsupervised Domain Adaptation

where:

Self-Paced Learning Policy Design The both k and kc in ST and CBST can be easily determined with a single SPL policy parameter p:

The Vanilla Self-Training (ST) Framework

The cost can be minimized via mixed integer programming, which leads to the following solution:

The Class-Balanced Self-Training (CBST) Framework

Again using mixed integer programming, one obtains the following solution:

Incorporating Spatial Priors (CBST-SP)