The availability of datasets pertaining to various fields has increased significantly in the past decade, but there still exists a problem in getting datasets pertaining to the medical field as most of the data needs to be confidential and there exists laws which ensure a patient's data privacy. Federated learning (FL) proves to solve this problem via a client-server architecture by enabling distributed training of clients, without any data exposure. In this paper, we apply the FedAvg (FederatedAveraging)  algorithm on the PathMNISTv2  dataset for predicting colorectal cancer. We also present a refined convolutional neural network (CNN) architecture for accurate predictions on the PathMNISTv2 dataset. We have studied the effects on IID (Independent and Identically Distributed) and Non-IID (Non-Identically Independently Distributed) distributions in a distributed environment. We have also compared these results with a centralized model and demonstrate that FedAvg achieves similar results in a distributed setting. We anticipate our study to enable additional healthcare studies driven by vast and diverse data, and illustrate the efficacy of FL at such magnitude and task complexity as a paradigm shift for multi-site partnerships, eliminating the need for data sharing.