Paper piano-Shadow analysis based touch interaction

Boga Vishal, K. Deepak Lawrence

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent years have marked a sharp increase in the number of ways in which people interact with electronic devices. Keyboards and computer mouse devices are not the only way users interact with their computers. This paper aims to investigate and demonstrate a human-computer interaction application in which a user can interact with the layout of a piano on a sheet of paper using computer vision. The image is acquired using a general-purpose camera from a single direction. The work presents details about methods of piano key detection and labelling, hand movement detection, fingertip detection and tracking, and touch interaction based on the shadow of the finger. K-means clustering is used to segment the hand and its shadow. Contour analysis is used in determining the shape and position of the keys. Shadow analysis aids in determining if the finger is touching the paper, even without any depth data from the camera. The detected key press then triggers a sound. The paper covers the details of the development of the virtual piano from concept to realization.

Original languageEnglish
Title of host publicationProceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-6
Number of pages6
Volume2018-March
ISBN (Electronic)9781538629895
DOIs
Publication statusPublished - 07-03-2018
Event2nd International Conference on Man and Machine Interfacing, MAMI 2017 - Bhubaneswar, India
Duration: 21-12-201723-12-2017

Conference

Conference2nd International Conference on Man and Machine Interfacing, MAMI 2017
CountryIndia
CityBhubaneswar
Period21-12-1723-12-17

Fingerprint

Cameras
Human computer interaction
Labeling
Computer vision
Acoustic waves

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Human-Computer Interaction
  • Computer Networks and Communications

Cite this

Vishal, B., & Lawrence, K. D. (2018). Paper piano-Shadow analysis based touch interaction. In Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017 (Vol. 2018-March, pp. 1-6). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MAMI.2017.8307890
Vishal, Boga ; Lawrence, K. Deepak. / Paper piano-Shadow analysis based touch interaction. Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017. Vol. 2018-March Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1-6
@inproceedings{2da9f14810f54a43ac7cd7902081de84,
title = "Paper piano-Shadow analysis based touch interaction",
abstract = "Recent years have marked a sharp increase in the number of ways in which people interact with electronic devices. Keyboards and computer mouse devices are not the only way users interact with their computers. This paper aims to investigate and demonstrate a human-computer interaction application in which a user can interact with the layout of a piano on a sheet of paper using computer vision. The image is acquired using a general-purpose camera from a single direction. The work presents details about methods of piano key detection and labelling, hand movement detection, fingertip detection and tracking, and touch interaction based on the shadow of the finger. K-means clustering is used to segment the hand and its shadow. Contour analysis is used in determining the shape and position of the keys. Shadow analysis aids in determining if the finger is touching the paper, even without any depth data from the camera. The detected key press then triggers a sound. The paper covers the details of the development of the virtual piano from concept to realization.",
author = "Boga Vishal and Lawrence, {K. Deepak}",
year = "2018",
month = "3",
day = "7",
doi = "10.1109/MAMI.2017.8307890",
language = "English",
volume = "2018-March",
pages = "1--6",
booktitle = "Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

Vishal, B & Lawrence, KD 2018, Paper piano-Shadow analysis based touch interaction. in Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017. vol. 2018-March, Institute of Electrical and Electronics Engineers Inc., pp. 1-6, 2nd International Conference on Man and Machine Interfacing, MAMI 2017, Bhubaneswar, India, 21-12-17. https://doi.org/10.1109/MAMI.2017.8307890

Paper piano-Shadow analysis based touch interaction. / Vishal, Boga; Lawrence, K. Deepak.

Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017. Vol. 2018-March Institute of Electrical and Electronics Engineers Inc., 2018. p. 1-6.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Paper piano-Shadow analysis based touch interaction

AU - Vishal, Boga

AU - Lawrence, K. Deepak

PY - 2018/3/7

Y1 - 2018/3/7

N2 - Recent years have marked a sharp increase in the number of ways in which people interact with electronic devices. Keyboards and computer mouse devices are not the only way users interact with their computers. This paper aims to investigate and demonstrate a human-computer interaction application in which a user can interact with the layout of a piano on a sheet of paper using computer vision. The image is acquired using a general-purpose camera from a single direction. The work presents details about methods of piano key detection and labelling, hand movement detection, fingertip detection and tracking, and touch interaction based on the shadow of the finger. K-means clustering is used to segment the hand and its shadow. Contour analysis is used in determining the shape and position of the keys. Shadow analysis aids in determining if the finger is touching the paper, even without any depth data from the camera. The detected key press then triggers a sound. The paper covers the details of the development of the virtual piano from concept to realization.

AB - Recent years have marked a sharp increase in the number of ways in which people interact with electronic devices. Keyboards and computer mouse devices are not the only way users interact with their computers. This paper aims to investigate and demonstrate a human-computer interaction application in which a user can interact with the layout of a piano on a sheet of paper using computer vision. The image is acquired using a general-purpose camera from a single direction. The work presents details about methods of piano key detection and labelling, hand movement detection, fingertip detection and tracking, and touch interaction based on the shadow of the finger. K-means clustering is used to segment the hand and its shadow. Contour analysis is used in determining the shape and position of the keys. Shadow analysis aids in determining if the finger is touching the paper, even without any depth data from the camera. The detected key press then triggers a sound. The paper covers the details of the development of the virtual piano from concept to realization.

UR - http://www.scopus.com/inward/record.url?scp=85047308758&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047308758&partnerID=8YFLogxK

U2 - 10.1109/MAMI.2017.8307890

DO - 10.1109/MAMI.2017.8307890

M3 - Conference contribution

VL - 2018-March

SP - 1

EP - 6

BT - Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Vishal B, Lawrence KD. Paper piano-Shadow analysis based touch interaction. In Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017. Vol. 2018-March. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1-6 https://doi.org/10.1109/MAMI.2017.8307890