Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning

Stuti Jindal, Sanjay Singh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

Images are the easiest medium through which people can express their emotions on social networking sites. Social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Significant progress has been made with this technology, however, there is little research focus on the picture sentiments. In this work, an image sentiment prediction framework is built with Convolutional Neural Networks (CNN). Specifically, this framework is pretrained on a large scale data for object recognition to further perform transfer learning. Extensive experiments were conducted on manually labeled Flickr image dataset. To make use of such labeled data, we employ a progressive strategy of domain specific fine tuning of the deep network. The results show that the proposed CNN training can achieve better performance in image sentiment analysis than competing networks.

Original languageEnglish
Title of host publicationProceedings - IEEE International Conference on Information Processing, ICIP 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages447-451
Number of pages5
ISBN (Electronic)9781467377584
DOIs
Publication statusPublished - 10-06-2016
Event2015 IEEE International Conference on Information Processing, ICIP 2015 - Pune, Maharashtra, India
Duration: 16-12-201519-12-2015

Conference

Conference2015 IEEE International Conference on Information Processing, ICIP 2015
CountryIndia
CityPune, Maharashtra
Period16-12-1519-12-15

Fingerprint

Image analysis
Tuning
Neural networks
Object recognition
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Signal Processing

Cite this

Jindal, S., & Singh, S. (2016). Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. In Proceedings - IEEE International Conference on Information Processing, ICIP 2015 (pp. 447-451). [7489424] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/INFOP.2015.7489424
Jindal, Stuti ; Singh, Sanjay. / Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. Proceedings - IEEE International Conference on Information Processing, ICIP 2015. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 447-451
@inproceedings{5fd53527daae4683ac44829ced94c527,
title = "Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning",
abstract = "Images are the easiest medium through which people can express their emotions on social networking sites. Social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Significant progress has been made with this technology, however, there is little research focus on the picture sentiments. In this work, an image sentiment prediction framework is built with Convolutional Neural Networks (CNN). Specifically, this framework is pretrained on a large scale data for object recognition to further perform transfer learning. Extensive experiments were conducted on manually labeled Flickr image dataset. To make use of such labeled data, we employ a progressive strategy of domain specific fine tuning of the deep network. The results show that the proposed CNN training can achieve better performance in image sentiment analysis than competing networks.",
author = "Stuti Jindal and Sanjay Singh",
year = "2016",
month = "6",
day = "10",
doi = "10.1109/INFOP.2015.7489424",
language = "English",
pages = "447--451",
booktitle = "Proceedings - IEEE International Conference on Information Processing, ICIP 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

Jindal, S & Singh, S 2016, Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. in Proceedings - IEEE International Conference on Information Processing, ICIP 2015., 7489424, Institute of Electrical and Electronics Engineers Inc., pp. 447-451, 2015 IEEE International Conference on Information Processing, ICIP 2015, Pune, Maharashtra, India, 16-12-15. https://doi.org/10.1109/INFOP.2015.7489424

Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. / Jindal, Stuti; Singh, Sanjay.

Proceedings - IEEE International Conference on Information Processing, ICIP 2015. Institute of Electrical and Electronics Engineers Inc., 2016. p. 447-451 7489424.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning

AU - Jindal, Stuti

AU - Singh, Sanjay

PY - 2016/6/10

Y1 - 2016/6/10

N2 - Images are the easiest medium through which people can express their emotions on social networking sites. Social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Significant progress has been made with this technology, however, there is little research focus on the picture sentiments. In this work, an image sentiment prediction framework is built with Convolutional Neural Networks (CNN). Specifically, this framework is pretrained on a large scale data for object recognition to further perform transfer learning. Extensive experiments were conducted on manually labeled Flickr image dataset. To make use of such labeled data, we employ a progressive strategy of domain specific fine tuning of the deep network. The results show that the proposed CNN training can achieve better performance in image sentiment analysis than competing networks.

AB - Images are the easiest medium through which people can express their emotions on social networking sites. Social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Significant progress has been made with this technology, however, there is little research focus on the picture sentiments. In this work, an image sentiment prediction framework is built with Convolutional Neural Networks (CNN). Specifically, this framework is pretrained on a large scale data for object recognition to further perform transfer learning. Extensive experiments were conducted on manually labeled Flickr image dataset. To make use of such labeled data, we employ a progressive strategy of domain specific fine tuning of the deep network. The results show that the proposed CNN training can achieve better performance in image sentiment analysis than competing networks.

UR - http://www.scopus.com/inward/record.url?scp=84979231286&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84979231286&partnerID=8YFLogxK

U2 - 10.1109/INFOP.2015.7489424

DO - 10.1109/INFOP.2015.7489424

M3 - Conference contribution

SP - 447

EP - 451

BT - Proceedings - IEEE International Conference on Information Processing, ICIP 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Jindal S, Singh S. Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. In Proceedings - IEEE International Conference on Information Processing, ICIP 2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 447-451. 7489424 https://doi.org/10.1109/INFOP.2015.7489424