A Survey and Performance Evaluation of Reinforcement Learning Based Spectrum Aware Routing in Cognitive Radio Ad Hoc Networks

Rashmi Naveen Raj, Ashalatha Nayak, M. Sathish Kumar

Research output: Contribution to journalArticle

Abstract

Cognitive radio technology is an assuring solution for under-utilization of licensed spectrum bands and overcrowding of unlicensed spectrum bands, in which secondary user is permitted to access the primary users’ spectrum in an opportunistic manner. Opportunistic access of the spectrum requires complex changes across all the layers of a network protocol stack. Cognitive radio has to be an autonomous agent in order to configure itself to dynamic spectrum environment. And, the characteristics of reinforcement learning, a subfield of artificial intelligence in which the agent learns the surrounding operating environment through continuous interaction and takes an optimum decision on the fly, is in compliance with features of self-organized cognitive radio ad hoc network. Therefore, reinforcement learning is an appropriate option for incorporating intelligence and self-adaptivity into cognitive radio. This paper provides a comprehensive survey on the application of reinforcement learning for efficient spectrum aware routing in cognitive radio ad hoc network. The preliminaries of cognitive radio ad hoc networks and reinforcement learning are first introduced, and a review is investigated in the proposed research area along with a discussion on open research challenges with an aim to promote research. From the survey, reinforcement learning incorporated cognitive radio can learn the unknown primary user network model and the learned model can be then used for finding a suitable route to meet the Quality of Service requirements. With this in mind, the paper also proposes a multi-objective reinforcement learning based spectrum aware routing protocol with an aim to maximize the probability of successful transmission using a minimum hop path. The simulated results prove the performance of the algorithm.

Original languageEnglish
JournalInternational Journal of Wireless Information Networks
DOIs
Publication statusAccepted/In press - 01-01-2019

Fingerprint

Reinforcement learning
Cognitive radio
Ad hoc networks
Autonomous agents
Routing protocols
Artificial intelligence
Quality of service
Network protocols

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Cite this

@article{4ef93421052e4d5482e8a5c28461ca75,
title = "A Survey and Performance Evaluation of Reinforcement Learning Based Spectrum Aware Routing in Cognitive Radio Ad Hoc Networks",
abstract = "Cognitive radio technology is an assuring solution for under-utilization of licensed spectrum bands and overcrowding of unlicensed spectrum bands, in which secondary user is permitted to access the primary users’ spectrum in an opportunistic manner. Opportunistic access of the spectrum requires complex changes across all the layers of a network protocol stack. Cognitive radio has to be an autonomous agent in order to configure itself to dynamic spectrum environment. And, the characteristics of reinforcement learning, a subfield of artificial intelligence in which the agent learns the surrounding operating environment through continuous interaction and takes an optimum decision on the fly, is in compliance with features of self-organized cognitive radio ad hoc network. Therefore, reinforcement learning is an appropriate option for incorporating intelligence and self-adaptivity into cognitive radio. This paper provides a comprehensive survey on the application of reinforcement learning for efficient spectrum aware routing in cognitive radio ad hoc network. The preliminaries of cognitive radio ad hoc networks and reinforcement learning are first introduced, and a review is investigated in the proposed research area along with a discussion on open research challenges with an aim to promote research. From the survey, reinforcement learning incorporated cognitive radio can learn the unknown primary user network model and the learned model can be then used for finding a suitable route to meet the Quality of Service requirements. With this in mind, the paper also proposes a multi-objective reinforcement learning based spectrum aware routing protocol with an aim to maximize the probability of successful transmission using a minimum hop path. The simulated results prove the performance of the algorithm.",
author = "{Naveen Raj}, Rashmi and Ashalatha Nayak and Kumar, {M. Sathish}",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s10776-019-00463-6",
language = "English",
journal = "International Journal of Wireless Information Networks",
issn = "1068-9605",
publisher = "Springer New York",

}

TY - JOUR

T1 - A Survey and Performance Evaluation of Reinforcement Learning Based Spectrum Aware Routing in Cognitive Radio Ad Hoc Networks

AU - Naveen Raj, Rashmi

AU - Nayak, Ashalatha

AU - Kumar, M. Sathish

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Cognitive radio technology is an assuring solution for under-utilization of licensed spectrum bands and overcrowding of unlicensed spectrum bands, in which secondary user is permitted to access the primary users’ spectrum in an opportunistic manner. Opportunistic access of the spectrum requires complex changes across all the layers of a network protocol stack. Cognitive radio has to be an autonomous agent in order to configure itself to dynamic spectrum environment. And, the characteristics of reinforcement learning, a subfield of artificial intelligence in which the agent learns the surrounding operating environment through continuous interaction and takes an optimum decision on the fly, is in compliance with features of self-organized cognitive radio ad hoc network. Therefore, reinforcement learning is an appropriate option for incorporating intelligence and self-adaptivity into cognitive radio. This paper provides a comprehensive survey on the application of reinforcement learning for efficient spectrum aware routing in cognitive radio ad hoc network. The preliminaries of cognitive radio ad hoc networks and reinforcement learning are first introduced, and a review is investigated in the proposed research area along with a discussion on open research challenges with an aim to promote research. From the survey, reinforcement learning incorporated cognitive radio can learn the unknown primary user network model and the learned model can be then used for finding a suitable route to meet the Quality of Service requirements. With this in mind, the paper also proposes a multi-objective reinforcement learning based spectrum aware routing protocol with an aim to maximize the probability of successful transmission using a minimum hop path. The simulated results prove the performance of the algorithm.

AB - Cognitive radio technology is an assuring solution for under-utilization of licensed spectrum bands and overcrowding of unlicensed spectrum bands, in which secondary user is permitted to access the primary users’ spectrum in an opportunistic manner. Opportunistic access of the spectrum requires complex changes across all the layers of a network protocol stack. Cognitive radio has to be an autonomous agent in order to configure itself to dynamic spectrum environment. And, the characteristics of reinforcement learning, a subfield of artificial intelligence in which the agent learns the surrounding operating environment through continuous interaction and takes an optimum decision on the fly, is in compliance with features of self-organized cognitive radio ad hoc network. Therefore, reinforcement learning is an appropriate option for incorporating intelligence and self-adaptivity into cognitive radio. This paper provides a comprehensive survey on the application of reinforcement learning for efficient spectrum aware routing in cognitive radio ad hoc network. The preliminaries of cognitive radio ad hoc networks and reinforcement learning are first introduced, and a review is investigated in the proposed research area along with a discussion on open research challenges with an aim to promote research. From the survey, reinforcement learning incorporated cognitive radio can learn the unknown primary user network model and the learned model can be then used for finding a suitable route to meet the Quality of Service requirements. With this in mind, the paper also proposes a multi-objective reinforcement learning based spectrum aware routing protocol with an aim to maximize the probability of successful transmission using a minimum hop path. The simulated results prove the performance of the algorithm.

UR - http://www.scopus.com/inward/record.url?scp=85074584918&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074584918&partnerID=8YFLogxK

U2 - 10.1007/s10776-019-00463-6

DO - 10.1007/s10776-019-00463-6

M3 - Article

AN - SCOPUS:85074584918

JO - International Journal of Wireless Information Networks

JF - International Journal of Wireless Information Networks

SN - 1068-9605

ER -