Vehicle counting is a process to estimate the road traffic density to assess the traffic conditions for intelligent transportation systems. With the extensive utilization of cameras in urban transport systems, the surveillance video has become a central data source. Also, real-time traffic management system has become popular recently due to the availability of handheld/mobile cameras and big-data analysis. In this work, we propose video-based vehicle counting method in a highway traffic video captured using handheld cameras. The processing of a video is achieved in three stages such as object detection by means of YOLO (You Only Look Once), tracking with correlation filter, and counting. YOLO attained remarkable outcome in the object detection area, and correlation filters achieved greater accuracy and competitive speed in tracking. Thus, we build multiple object tracking with correlation filters using the bounding boxes generated by the YOLO framework. Experimental analysis using real video sequences shows that the proposed method can detect, track and count the vehicles accurately.