Computer vision-based technologies can be effectively adopted to enhance the performance and productivity of aquaculture industries. Application of these technologies can ease the life of fish farmers and improve the harvest of aquaculture. Fishes are much susceptible to their environment. Small changes in the water quality parameter can increase the mortality rate. Fishes are also known to show abnormal behaviour patterns when experiencing stress. Early detection of these anomalous patterns can avoid commercial losses for aqua fish farmers. Culturing of fish like Sillago-sihama is a tedious and risky task as it is highly sensitive to its environment. On the other hand, it has a high nutrient and commercial value. To this end, an attempt is made to develop a decision support system for identifying abnormal behaviour patterns of Sillago-sihama and thereby assisting the fish farmers to improve productivity. The proposed research detects three behavioural patterns of Sillago-sihama viz. swimming at the surface, no movement and frantic movement patterns. This work proposes a pattern analysis and behaviour identification model using the motion information obtained from tracking by detection method. Extensive experimental results show that the novel approach is reliable in detecting different patterns of Sillago-sihama.