Eman Hassan

Unsupervised Domain Adaptation using Generative Models and Self-ensembling

Transferring knowledge across different datasets is an important approach to successfully train deep models with a small-scale target dataset or when few labeled instances are available. In this paper, we aim at developing a model that can generalize across multiple domain shifts, so that this model can adapt from a single source to multiple tar- gets. This can be achieved by randomizing the generation of the data of various styles to mitigate the domain mismatch. First, we present a new adaptation to the CycleGAN model to produce stochastic style transfer between two image batches of different domains. Second, we enhance the classifier performance by using a self-ensembling technique with a teacher and student model to train on both original and generated data. Finally, we present experimental results on three datasets Office-31, Office-Home, and Visual Domain adaptation. The results suggest that self-ensembling is better than simple data augmentation with the newly generated data and a single model trained this way can have the best performance across all different transfer tasks.

Archive Paper: https://arxiv.org/abs/1812.00479 »

Cross-domain Generative Models Applied to Cartoon Series

We investigate Generative Adversarial Networks (GANs) to model one particular kind of image: frames from TV cartoons. Cartoons are particularly interesting because their visual appearance emphasizes the important semantic information about a scene while abstracting out the less important details, but each cartoon series has a distinctive artistic style that performs this abstraction in different ways. We consider a dataset consisting of images from two popular television cartoon series, Family Guy and The Simpsons.

We examine the ability of GANs to generate images from each of these two domains, when trained independently as well as on both domains jointly. We find that generative models may be capable of finding semantic-level correspondences between these two image domains despite the unsupervised setting, i.e. even when the training data does not give labeled alignments between them.

Archive Paper: https://arxiv.org/abs/1710.00755 »

Cartoon Summery Generation for Ego-centric videos

This work represents a fun way to generate a summarization of life logging data. The main idea is that objects always convey many information about the scene and the activity that is happening, we use mutual information to mine for important object to each activity, then we use that object area to sample frames representing the activity. Finally, we render the summary in a cartoon-like way by background cartooning and object replacements with clip arts. This could produce a possibly nice pleasing cartoon summary for an ego-centric life logging video.

Image Inpainting Based on image segmentation and segment classification

We present a new inpainting algorithm that is based on image segmentation and segment classification. First, we employ the mean shift algorithm to segment the input image. Then, we divide the original inpainting problem to be either one of the two problems: Large Segment Inpainting problem or Non-uniform Segments inpainting problem.

The reason we do that is that human eye is more discerning to the errors in the structure and texture propagation of a large-uniform regions with less details while it is less discerning to errors in non-uniform regions with more details.

We propose a novel algorithm for each one of the problems- Large Segment Inpainting and Non- uniform Segments inpainting- according to the main features of each one. The experimental results show the advantage of our technique which produces output images with better perceived visual quality.