Android Malware Detection using Deep Learning based Feature Fusion and Ensemble Learning Models through Image Representations

Malware detection has become a serious task because of rapid development of malicious software and its evolving patterns. This paper presents a comprehensive framework for malware classification by integrating deep learning, handcrafted feature extraction and machine learning techniques. Initially, a CNN is employed as a baseline model, followed by transfer learning models, including DenseNet and MobileNet, to enhance feature representation. To further boost performance, several handcrafted and deep features, including color histogram, Gabor, GLCM, HOG, LBP, ORB, SIFT, wavelet transform features and pre-trained deep features are extracted and combined to create a hybrid feature set. These features are then used to train a number of ML classifiers such as Random Forest, SVM, Logistic Regression, XGBoost, AdaBoost, Gradient Boosting, LightGBM and CatBoost. Among these, XGBoost achieved superior performance among individual models. Moreover, ensemble methods like voting and stacking classifiers are used to boost robustness and accuracy. The stacking classifier utilizing the metaclassifier Random Forest gave the best results of 91.6% accuracy. The results highlight the effectiveness and scalability of the proposed hybrid feature-based ensemble approach, which improves the performance of standalone deep learning models in the field of malware detection.