Electroencephalography (EEG) clinical tool is widely used for the diagnosis of epilepsy. This tool is able to capture the electrical activity of the brain. Automated computer aided diagnosis (CAD) systems, based on deep learning have gaining attention as Conventional EEG analysis are time-consuming as it is based on neurologists manually inspecting to identify epileptiform abnormalities. The conventional method is also affected by artifacts, and subjected to inter observer variability. In the current work, we classify the EEG using a one dimensional convolutional neural network (CNN) having 13 layers and expand this CNN with a controlled ablation framework (A0-A6). A single channel EEG segments from the Bonn University EEG dataset are used for this paper, with preprocessing the dataset, which included fixed length segmentation (4097 samples) and per segment z score normalization, without feature extraction. By plusging the best of all the mentioned models, with residual learning, data augmentation, attention mechanism, multi domain feature fusion, self supervised pretraining, is the best performing model, which yields the highest classification performance under ten fold cross validation. The candidates of the variants are compared with each other by applying the same training setup with Adam optimizer and stratified 10 fold cross validation. Results demonstrate that both the reproduced baseline CNN model and the proposed improved models, obtain 96.00% ± 2.49% accuracy, respectively, thus the proposed improved CNN models boost the performance of baseline CNN model. The model with residual learning, data augmentation and attention obtained the best classification accuracy, sensitivity and specificity, which was 100%, 100% and 100% respectively. 99.33% ± 1.33% Accuracy with Multi Domain Fusion and Self Supervised Pretraining. The proposed framework is reproducible, modular and appropriate for automatic classification of EEG based epileptic signal.