Symbolic verification of web crawler functionality and its properties

Keerthi S. Shetty, Swaraj Bhat, Sanjay Singh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Now a days people use search engines every now and then to retrieve documents from the Web. Web crawling is the process by which a search engine gather pages from the Web to index them and support a search engine. Web crawlers are the heart of search engines. Web crawlers continuously keep on crawling the web and find any new web pages that have been added to the web, pages that have been removed from the web. Due to growing and dynamic nature of the web, it has become a challenge to traverse through all URLs in the web documents and to handle these URLs. The entire crawling process may be viewed as traversing a web graph. The aim of this paper is to model check the crawling process and crawler properties using a symbolic model checker tool called NuSMV. The basic operation of a hypertext crawler and the crawler properties has been modeled in terms of CTL specification and it is observed that the system takes care of all the constraints by satisfying all the specifications.

Original languageEnglish
Title of host publication2012 International Conference on Computer Communication and Informatics, ICCCI 2012
DOIs
Publication statusPublished - 27-03-2012
Event2012 International Conference on Computer Communication and Informatics, ICCCI 2012 - Coimbatore, India
Duration: 10-01-201212-01-2012

Conference

Conference2012 International Conference on Computer Communication and Informatics, ICCCI 2012
CountryIndia
CityCoimbatore
Period10-01-1212-01-12

Fingerprint

World Wide Web
Search engines
Websites
Specifications
Web crawler

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems

Cite this

Shetty, K. S., Bhat, S., & Singh, S. (2012). Symbolic verification of web crawler functionality and its properties. In 2012 International Conference on Computer Communication and Informatics, ICCCI 2012 [6158649] https://doi.org/10.1109/ICCCI.2012.6158649
Shetty, Keerthi S. ; Bhat, Swaraj ; Singh, Sanjay. / Symbolic verification of web crawler functionality and its properties. 2012 International Conference on Computer Communication and Informatics, ICCCI 2012. 2012.
@inproceedings{74766a5c04694674955b96e6671a2774,
title = "Symbolic verification of web crawler functionality and its properties",
abstract = "Now a days people use search engines every now and then to retrieve documents from the Web. Web crawling is the process by which a search engine gather pages from the Web to index them and support a search engine. Web crawlers are the heart of search engines. Web crawlers continuously keep on crawling the web and find any new web pages that have been added to the web, pages that have been removed from the web. Due to growing and dynamic nature of the web, it has become a challenge to traverse through all URLs in the web documents and to handle these URLs. The entire crawling process may be viewed as traversing a web graph. The aim of this paper is to model check the crawling process and crawler properties using a symbolic model checker tool called NuSMV. The basic operation of a hypertext crawler and the crawler properties has been modeled in terms of CTL specification and it is observed that the system takes care of all the constraints by satisfying all the specifications.",
author = "Shetty, {Keerthi S.} and Swaraj Bhat and Sanjay Singh",
year = "2012",
month = "3",
day = "27",
doi = "10.1109/ICCCI.2012.6158649",
language = "English",
isbn = "9781457715822",
booktitle = "2012 International Conference on Computer Communication and Informatics, ICCCI 2012",

}

Shetty, KS, Bhat, S & Singh, S 2012, Symbolic verification of web crawler functionality and its properties. in 2012 International Conference on Computer Communication and Informatics, ICCCI 2012., 6158649, 2012 International Conference on Computer Communication and Informatics, ICCCI 2012, Coimbatore, India, 10-01-12. https://doi.org/10.1109/ICCCI.2012.6158649

Symbolic verification of web crawler functionality and its properties. / Shetty, Keerthi S.; Bhat, Swaraj; Singh, Sanjay.

2012 International Conference on Computer Communication and Informatics, ICCCI 2012. 2012. 6158649.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Symbolic verification of web crawler functionality and its properties

AU - Shetty, Keerthi S.

AU - Bhat, Swaraj

AU - Singh, Sanjay

PY - 2012/3/27

Y1 - 2012/3/27

N2 - Now a days people use search engines every now and then to retrieve documents from the Web. Web crawling is the process by which a search engine gather pages from the Web to index them and support a search engine. Web crawlers are the heart of search engines. Web crawlers continuously keep on crawling the web and find any new web pages that have been added to the web, pages that have been removed from the web. Due to growing and dynamic nature of the web, it has become a challenge to traverse through all URLs in the web documents and to handle these URLs. The entire crawling process may be viewed as traversing a web graph. The aim of this paper is to model check the crawling process and crawler properties using a symbolic model checker tool called NuSMV. The basic operation of a hypertext crawler and the crawler properties has been modeled in terms of CTL specification and it is observed that the system takes care of all the constraints by satisfying all the specifications.

AB - Now a days people use search engines every now and then to retrieve documents from the Web. Web crawling is the process by which a search engine gather pages from the Web to index them and support a search engine. Web crawlers are the heart of search engines. Web crawlers continuously keep on crawling the web and find any new web pages that have been added to the web, pages that have been removed from the web. Due to growing and dynamic nature of the web, it has become a challenge to traverse through all URLs in the web documents and to handle these URLs. The entire crawling process may be viewed as traversing a web graph. The aim of this paper is to model check the crawling process and crawler properties using a symbolic model checker tool called NuSMV. The basic operation of a hypertext crawler and the crawler properties has been modeled in terms of CTL specification and it is observed that the system takes care of all the constraints by satisfying all the specifications.

UR - http://www.scopus.com/inward/record.url?scp=84863343721&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863343721&partnerID=8YFLogxK

U2 - 10.1109/ICCCI.2012.6158649

DO - 10.1109/ICCCI.2012.6158649

M3 - Conference contribution

SN - 9781457715822

BT - 2012 International Conference on Computer Communication and Informatics, ICCCI 2012

ER -

Shetty KS, Bhat S, Singh S. Symbolic verification of web crawler functionality and its properties. In 2012 International Conference on Computer Communication and Informatics, ICCCI 2012. 2012. 6158649 https://doi.org/10.1109/ICCCI.2012.6158649