Handwritten digit recognition is a well-studied pattern recognition problem. Most of the techniques, reported in the literature, have been concentrated on designing several feature vectors which represent the digits in a better way. But, at times, such attempt not only increases the dimensionality of extracted feature vector but also suffers from having irrelevant and/or redundant features. To address this, in the present work, a recently introduced Particle Swarm Optimization (PSO) based feature selection method has been applied with suitable modifications. For the course of this experiment, we have confined ourselves to a newly employed feature vector for handwritten digit recognition, namely DAISY feature descriptor. The proposed feature selection method is tested on three handwritten digit databases written in Bangla, Devanagari and Roman scripts. The experimental results show that significant amount of feature dimension is reduced without compromising on recognition accuracy. Comparison of the present feature selection method with two of its ancestors also reveals that the present method outperforms the others.