Exploring Techniques for Photo-realistic Image Generation from 3D Models-A Deep Learning Approach

Pranoy Ghosh, Krithika M. Pai, M. M. Manohara Pai, Ujjwal Verma, Frederic Rivet, Abhishek Roul, V. Venugopal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The product companies are going towards Digital Transformation for digital customer experience. This is effective for marketing and sales specially during the pandemic. For the product visualization, it is necessary to create multiple 2D views of a product which is a cumbersome and time-consuming process especially when there are copious number of products. Traditional rendering techniques and image augmentation for dataset generation either take quite a lot of time or do not yield photo-realistic images. The recent progress in generative models and 3D deep learning provide a promising avenue in this regard. This paper explores different techniques for rendering photo-realistic 2D images from 3D CAD models. This study compares the performance of three different models pix2pix, Pytorch3D and SynSin based on three different approaches namely image-to-image translation, mesh rendering, and view synthesis respectively to generate photo-realistic images. The models are tested using the dataset provided by Schneider Electric. Results demonstrate that Pytorch3D is the better model to generate photo-realistic images. These approaches can emerge as the initial steps for digital twin technology.

Original languageEnglish
Title of host publication2021 IEEE Mysore Sub Section International Conference, MysuruCon 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages697-702
Number of pages6
ISBN (Electronic)9780738146621
DOIs
Publication statusPublished - 2021
Event1st IEEE Mysore Sub Section International Conference, MysuruCon 2021 - Hassan, India
Duration: 24-10-202125-10-2021

Publication series

Name2021 IEEE Mysore Sub Section International Conference, MysuruCon 2021

Conference

Conference1st IEEE Mysore Sub Section International Conference, MysuruCon 2021
Country/TerritoryIndia
CityHassan
Period24-10-2125-10-21

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Engineering (miscellaneous)
  • Computational Mechanics
  • Control and Optimization
  • Instrumentation

Fingerprint

Dive into the research topics of 'Exploring Techniques for Photo-realistic Image Generation from 3D Models-A Deep Learning Approach'. Together they form a unique fingerprint.

Cite this