Meta-learner with sparsified backpropagation

Rohan Paithankar, Aayushi Verma, Manish Agnihotri, Sanjay Singh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In todays world, Deep Learning is an area of research that has ever increasing applications. It deals with the use of neural networks to bring improvements in areas like speech recognition, computer vision, natural language processing and several automated systems. Training deep neural networks involves careful selection of appropriate training examples, tuning of hyperparameters and scheduling step sizes, finding a proper combination of all these is a tedious and time-consuming task. In the recent times, a few learning-to-learn models have been proposed, that can learn automatically. The time and accuracy of the model are exceedingly important. A technique named meProp was proposed to accelerate Deep Learning with reduced over-fitting. meProp is a method that proposes a sparsified back propagation method which reduces the computational cost. In this paper, we propose an application of meProp to the learning-to-learn models to focus on learning of the most significant parameters which are consciously chosen. We demonstrate improvement in accuracy of the learning-to-learn model with the proposed technique and compare the performance with that of the unmodified learning-to-learn model.

Original languageEnglish
Title of host publicationProceedings of the 9th International Conference On Cloud Computing, Data Science and Engineering, Confluence 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages315-319
Number of pages5
ISBN (Electronic)9781538659335
DOIs
Publication statusPublished - 01-01-2019
Event9th International Conference On Cloud Computing, Data Science and Engineering, Confluence 2019 - Uttar Pradesh, India
Duration: 10-01-201911-01-2019

Publication series

NameProceedings of the 9th International Conference On Cloud Computing, Data Science and Engineering, Confluence 2019

Conference

Conference9th International Conference On Cloud Computing, Data Science and Engineering, Confluence 2019
CountryIndia
CityUttar Pradesh
Period10-01-1911-01-19

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Computer Networks and Communications
  • Software
  • Information Systems

Fingerprint Dive into the research topics of 'Meta-learner with sparsified backpropagation'. Together they form a unique fingerprint.

  • Cite this

    Paithankar, R., Verma, A., Agnihotri, M., & Singh, S. (2019). Meta-learner with sparsified backpropagation. In Proceedings of the 9th International Conference On Cloud Computing, Data Science and Engineering, Confluence 2019 (pp. 315-319). [8776608] (Proceedings of the 9th International Conference On Cloud Computing, Data Science and Engineering, Confluence 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CONFLUENCE.2019.8776608