In the world, India is the second biggest producer of peanuts or groundnuts, and it is also our country’s major oilseed crop. In India, the existing peanuts crop varieties are GAUG-1, Kuber, Amber, PG-1, BG-1, T-64, GAUG-10, BG-2, Chandra, Kadri-2, Chitra, Kadri-3, Prakash, T-28, Kaushal, etc. Presently, peanuts are having only 75–80% of India’s average market value. Because, the peanuts kernel quality assessment, as well as identifying varieties, are done manually by skilled labors, which leads costly. In this research, an affordable method is proposed to assess the peanuts kernel quality and identifying the different varieties quickly with undamaged, repeatability with low cost, and accurately with high distinguishing rate. Also, to meet the quality of peanuts kernel as per the international market standards and to increase the income of the former. The proposed system relies on computer vision and machine learning. The obtained overall accuracies were K-nearest neighbors (93.33%) and Support vector machine (93.82%). These percentages are discriminating peanuts variety as the best predictive model.