Adaptive Avatar Customization for Immersive Experiences

In this thesis, we aim to perform character stylization in a fully automatic manner. Personalizable characters have been present in games for several decades, allowing users to stylize (alter) the appearance of their faces, bodies, or clothes of the character they incarnate or interact with. In the case of movies, with only tens of actors compared to thousands or millions of gamers, personalizing characters does not require automation and can be seen instead as a cost saving factor, since realistic hand crafted personalized embodiments are limited to a few per high budget movies (e.g. The Lord of the Rings – 2001, Avatar – 2009 1.1). The concept of having a whole digital double is a future strongly pushed forward by commercial products such as the Metaverse. As modern day virtual embodiments grow closer and closer to ourselves, and as we reach times where entirely virtual characters can look realistic on screen, there is a growing interest in being able to personalize your own embodiment. This means that there is a need of a stylized character resembling a reference person, while conserving some core identity features, to let them be recognizable. In this thesis, we target a level of realism close to realist video games, aiming to provide a strong baseline which could be used for a game or VR application, or fine-tuned and worked on by artists to be used in a context requiring higher photorealism. The use of neural network based technology has known considerable growth during these last few years. In the artistic world it has been used both as a tool to support creativity, and as a way towards art creation by itself, through the use of generative networks. Neural networks have been heavily leveraged and designed for content stylization. Stylization – or style transfer – can be characterized as the mapping of one distribution (as in, a function giving all possible values of some data, with their occurrence frequency) towards another, where both distributions share part of their information. In the literature, style transfer has been mostly modeled as the separation of data into content (what is shared between distributions, identity in the case of faces), and style (everything that is domain specific).