How much realism is necessary in data augmentation techniques for traffic light detection

Training a deep learning model requires large dataset. Collecting and labeling data can be very costly and sometimes infeasible. That is why data augmentation can play an important role in providing a source of generating new labeled data to train a deep learning algorithm. The importance of realism and semantic consistency in data augmentation can vary depending on the type of the data set (indoor vs outdoor scenes) and the type of the problem statement.

In our case we care about Traffic light detection in urban road scenes. Traffic light detection is very challenging as the object can be very far, occluded and in many cases doesn’t have very distinct features from the background.

In our work we conducted a study to find out the effectiveness of different methods of data augmentation on traffic light detection compared to real data collection, how different data augmentation techniques correlates with the realism of scene generation and whether or not GAN models can be used to enhance the quality of the data augmentation. Finally, we suggest new directions in using generative models to enhance small objects detection depending only on the semantics not the object features.

Unsupervised Domain Adaptation using Generative Models and Self-ensembling

Transferring knowledge across different datasets is an important approach to successfully train deep models with a small-scale target dataset or when few labeled instances are available. In this paper, we aim at developing a model that can generalize across multiple domain shifts, so that this model can adapt from a single source to multiple tar- gets. This can be achieved by randomizing the generation of the data of various styles to mitigate the domain mismatch. First, we present a new adaptation to the CycleGAN model to produce stochastic style transfer between two image batches of different domains. Second, we enhance the classifier performance by using a self-ensembling technique with a teacher and student model to train on both original and generated data. Finally, we present experimental results on three datasets Office-31, Office-Home, and Visual Domain adaptation. The results suggest that self-ensembling is better than simple data augmentation with the newly generated data and a single model trained this way can have the best performance across all different transfer tasks.

Archive Paper: https://arxiv.org/abs/1812.00479 »

Cross-domain Generative Models Applied to Cartoon Series

We investigate Generative Adversarial Networks (GANs) to model one particular kind of image: frames from TV cartoons. Cartoons are particularly interesting because their visual appearance emphasizes the important semantic information about a scene while abstracting out the less important details, but each cartoon series has a distinctive artistic style that performs this abstraction in different ways. We consider a dataset consisting of images from two popular television cartoon series, Family Guy and The Simpsons.

We examine the ability of GANs to generate images from each of these two domains, when trained independently as well as on both domains jointly. We find that generative models may be capable of finding semantic-level correspondences between these two image domains despite the unsupervised setting, i.e. even when the training data does not give labeled alignments between them.

Archive Paper: https://arxiv.org/abs/1710.00755 »

Can Privacy Be Satisfying

Pervasive photo sharing in online social media platforms can cause unintended privacy violations when elements of an image reveal sensitive information. Prior studies have identified image obfuscation methods (e.g., blurring) to enhance privacy, but many of these methods adversely affect viewers’ satisfaction with the photo, which may cause people to avoid using them. In this paper, we study the novel hypothesis that it may be possible to restore viewers’ satisfaction by ‘boosting’ or enhancing the aesthetics of an obscured image, thereby compensating for the negative effects of a privacy transform. Using a between-subjects online experiment, we studied the effects of three artistic transformations on images that had objects obscured using three popular obfuscation methods validated by prior research. Our findings suggest that using artistic transformations can mitigate some negative effects of obfuscation methods, but more exploration is needed to retain viewer satisfaction.

Conference Paper:https://www.cs.indiana.edu/~kapadia/papers/hasan-chi-19.pdf»

Viewer Experience of Obscuring Scene Elements in Photos to Enhance Privacy

With the rise of digital photography and social networking, people are sharing personal photos online at an unprecedented rate. In addition to their main subject matter, photographs often capture various incidental information that could harm people’s privacy. While blurring and other image filters may help obscure private content, they also often affect the utility and aesthetics of the photos, which is important since images shared in social media are mainly for human consumption. Existing studies of privacy-enhancing image filters either primarily focus on obscuring faces, or do not systematically study how filters affect image utility. To understand the trade-offs when obscuring various sensitive aspects of images, we study eleven filters applied to obfuscate twenty different objects and attributes, and evaluate how effectively they protect privacy and preserve image quality for human viewers.

Conference Paper:www.cs.indiana.edu/~kapadia/papers/hasan-chi-18.pdf»

Cartoon Summery Generation for Ego-centric videos

This work represents a fun way to generate a summarization of life logging data. The main idea is that objects always convey many information about the scene and the activity that is happening, we use mutual information to mine for important object to each activity, then we use that object area to sample frames representing the activity. Finally, we render the summary in a cartoon-like way by background cartooning and object replacements with clip arts. This could produce a possibly nice pleasing cartoon summary for an ego-centric life logging video.

Image Inpainting Based on image segmentation and segment classification

We present a new inpainting algorithm that is based on image segmentation and segment classification. First, we employ the mean shift algorithm to segment the input image. Then, we divide the original inpainting problem to be either one of the two problems: Large Segment Inpainting problem or Non-uniform Segments inpainting problem.

The reason we do that is that human eye is more discerning to the errors in the structure and texture propagation of a large-uniform regions with less details while it is less discerning to errors in non-uniform regions with more details.

We propose a novel algorithm for each one of the problems- Large Segment Inpainting and Non- uniform Segments inpainting- according to the main features of each one. The experimental results show the advantage of our technique which produces output images with better perceived visual quality.