REVIEW OF FEATURE SELECTION METHODS USING OPTIMIZATION ALGORITHM
Review paper for optimization algorithm
DOI:
https://doi.org/10.25156/ptj.v12n2y2022.pp203-214Keywords:
Features selection, Filter process, Wrappers, embedded, Metaheuristic algorithmAbstract
Many works have been done to reduce complexity in terms of time and memory space. The feature selection process is one of the strategies to reduce system complexity and can be defined as a process of selecting the most important feature among feature space. Therefore, the most useful features will be kept, and the less useful features will be eliminated. In the fault classification and diagnosis field, feature selection plays an important role in reducing dimensionality and sometimes might lead to having a high classification rate. In this paper, a comprehensive review is presented about feature selection processing and how it can be done. The primary goal of this research is to examine all of the strategies that have been used to highlight the (selection) selected process, including filter, wrapper, Meta-heuristic algorithm, and embedded. Review of Nature-inspired algorithms that have been used for features selection is more focused such as particle swarm, Grey Wolf, Bat, Genetic, wale, and ant colony algorithm. The overall results confirmed that the feature selection approach is important in reducing the complexity of any model-based machine learning algorithm and may sometimes result in improved performance of the simulated model.
Downloads
References
Abdullah, J. M., & Ahmed, T. (2019). Fitness dependent optimizer: inspired by the bee swarming reproductive process. IEEE Access, 7, 43473-43486. https://doi.org/10.1109/ACCESS.2019.2907012.
Aghdam, M. H., Ghasem-Aghaee, N., & Basiri, M. E. (2009). Text feature selection using ant colony optimization. Expert Systems with Applications, 36(3 PART 2), 6843–6853. https://doi.org/10.1016/j.eswa.2008.08.022.
Ahmed, A.-A. (2005). Feature subset selection using ant colony optimization. International Journal of Computational, 2(1), 53–58. http://epress.lib.uts.edu.au/research/handle/10453/6002.
Ahmed, A. M., Rashid, T. A., & Saeed, S. A. M. (2020). Cat swarm optimization algorithm: a survey and performance evaluation. Computational intelligence and neuroscience, 2020.
Bozorg-Haddad, O. (2018). Studies in Computational Intelligence - Advanced Optimization by Nature-Inspired Algorithms. In Studies in Computational Intelligence.
Chen, L. F., Su, C. T., & Chen, K. H. (2012). An improved particle swarm optimization for feature selection. Intelligent Data Analysis, 16(2), 167–182. https://doi.org/10.3233/IDA-2012-0517.
Chen, Y., Miao, D., & Wang, R. (2010). A rough set approach to feature selection based on ant colony optimization. Pattern Recognition Letters, 31(3), 226–233. https://doi.org/10.1016/j.patrec.2009.10.013.
Cremona, C., Yang, S., & Dieleman, L. (2009). Overview of feature extraction techniques. IOMAC 2009 - 3rd International Operational Modal Analysis Conference, September, 381–390. https://doi.org/10.2139/ssrn.3568122.
Dash, M., & Liu, H. (1997). Feature selection for classification. Intelligent Data Analysis, 1(3), 131–156. https://doi.org/10.3233/IDA-1997-1302.
Dash, Manoranjan, & Liu, H. (2003). Consistency-based search in feature selection. Artificial Intelligence, 151(1–2), 155–176. https://doi.org/10.1016/S0004-3702(03)00079-1.
Diao, R., & Shen, Q. (2015). Nature inspired feature selection meta-heuristics. Artificial Intelligence Review, 44(3), 311–340. https://doi.org/10.1007/s10462-015-9428-8.
Emary, E., Zawbaa, H. M., & Hassanien, A. E. (2016). Binary grey wolf optimization approaches for feature selection. Neurocomputing, 172, 371–381. https://doi.org/10.1016/j.neucom.2015.06.083.
Faris, H., Aljarah, I., Al-Betar, M. A., & Mirjalili, S. (2018). Grey wolf optimizer: a review of recent variants and applications. Neural Computing and Applications, 30(2), 413–435. https://doi.org/10.1007/s00521-017-3272-5.
Forsati, R., Moayedikia, A., Keikha, A., & Shamsfard, M. (2012). A Novel Approach for Feature Selection based on the Bee Colony Optimization. International Journal of Computer Applications, 43(8), 13–16. https://doi.org/10.5120/6122-8329.
Guo, H., Jack, L. B., & Nandi, A. K. (2005). Feature generation using genetic programming with application to fault classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 35(1), 89–99. https://doi.org/10.1109/TSMCB.2004.841426.
Hajnayeb, A., Ghasemloonia, A., Khadem, S. E., & Moradi, M. H. (2011). Application and comparison of an ANN-based feature selection method and the genetic algorithm in gearbox fault diagnosis. Expert Systems with Applications, 38(8), 10205–10209. https://doi.org/10.1016/j.eswa.2011.02.065.
Hancer, E., Xue, B., Zhang, M., Karaboga, D., & Akay, B. (2018). Pareto front feature selection based on artificial bee colony optimization. Information Sciences, 422, 462–479. https://doi.org/10.1016/j.ins.2017.09.028.
Jain, I., Jain, V. K., & Jain, R. (2018). Correlation feature selection based improved-Binary Particle Swarm Optimization for gene selection and cancer classification. Applied Soft Computing, 62, 203–215. https://doi.org/10.1016/j.asoc.2017.09.038.
Jeyasingh, S., & Veluchamy, M. (2017). Modified bat algorithm for feature selection with the Wisconsin Diagnosis Breast Cancer (WDBC) dataset. Asian Pacific Journal of Cancer Prevention, 18(5), 1257–1264. https://doi.org/10.22034/APJCP.2017.18.5.1257.
Jović, A., Brkić, K., & Bogunović, N. (2015). A review of feature selection methods with applications. 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2015 - Proceedings, 1200–1205. https://doi.org/10.1109/MIPRO.2015.7160458.
KARABOGA, D. (2005). AN IDEA BASED ON HONEY-BEE SWARM FOR NUMERICAL OPTIMIZATION. Globalisasi Dan Kebidayaan Lokal: Suatu Dialektika Menuju Indonesia Baru, 7(2), 147–173. http://dx.doi.org/10.1016/j. Karaboga, D., & Akay, B. (2009). A comparative study of Artificial Bee Colony algorithm. Applied Mathematics and Computation, 214(1), 108–132. https://doi.org/10.1016/j.amc.2009.03.090.
Khalid, S., Khalil, T., & Nasreen, S. (2014). A survey of feature selection and feature extraction techniques in machine learning. Proceedings of 2014 Science and Information Conference, SAI 2014, August 2014, 372–378. https://doi.org/10.1109/SAI.2014.6918213.
Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97(1–2), 273–324. https://doi.org/10.1016/s0004-3702(97)00043-x.
Lin, K. C., Huang, Y. H., Hung, J. C., & Lin, Y. T. (2015). Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Cat Swarm Optimization. International Journal of Distributed Sensor Networks, 2015. https://doi.org/10.1155/2015/365869.
Lin, K. C., Zhang, K. Y., Huang, Y. H., Hung, J. C., & Yen, N. (2016). Feature selection based on an improved cat swarm optimization algorithm for big data classification. Journal of Supercomputing, 72(8), 3210–3221. https://doi.org/10.1007/s11227-016-1631-0.
Mafarja, M. M., & Mirjalili, S. (2017). Hybrid Whale Optimization Algorithm with simulated annealing for feature selection. Neurocomputing, 260, 302–312. https://doi.org/10.1016/j.neucom.2017.04.053.
Mafarja, M., & Mirjalili, S. (2018). Whale optimization approaches for wrapper feature selection. Applied Soft Computing, 62, 441–453. https://doi.org/10.1016/j.asoc.2017.11.006.
Mirjalili, S., & Lewis, A. (2016). The Whale Optimization Algorithm. Advances in Engineering Software, 95, 51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008.
Muhammed, D. A., Saeed, S. A. M., & Rashid, T. A. (2020). Improved Fitness-Dependent Optimizer Algorithm. IEEE Access, 8, 19074–19088. https://doi.org/10.1109/ACCESS.2020.2968064.
Nakamura, R. Y. M., Pereira, L. A. M., Costa, K. A., Rodrigues, D., Papa, J. P., & Yang, X. S. (2012). BBA: A binary bat algorithm for feature selection. Brazilian Symposium of Computer Graphic and Image Processing, 291–297. https://doi.org/10.1109/SIBGRAPI.2012.47.
Niu, P., Niu, S., liu, N., & Chang, L. (2019). The defect of the Grey Wolf optimization algorithm and its verification method. Knowledge-Based Systems, 171, 37–43. https://doi.org/10.1016/j.knosys.2019.01.018.
Oscar Cordon, farncisco Herrera, T. (2002). 1-Cordon-Herrera-Stuetzle.
Rashid, T. A., Fattah, P., & Awla, D. K. (2018). Using accuracy measure for improving the training of LSTM with metaheuristic algorithms. Procedia Computer Science, 140, 324-333.
Sakri, S. B., Abdul Rashid, N. B., & Muhammad Zain, Z. (2018). Particle Swarm Optimization Feature Selection for Breast Cancer Recurrence Prediction. IEEE Access, 6, 29637–29647. https://doi.org/10.1109/ACCESS.2018.2843443.
Sharawi, M., Zawbaa, H. M., Emary, E., & Hossamzawbaagmailcom, E. (n.d.). Feature Selection Approach Based on Whale Optimization Algorithm.
Sharma, A., Amarnath, M., & Kankar, P. K. (2016). Feature extraction and fault severity classification in ball bearings. JVC/Journal of Vibration and Control, 22(1), 176–192. https://doi.org/10.1177/1077546314528021.
Shrivastava, P., Shukla, A., Vepakomma, P., Bhansali, N., & Verma, K. (2017). A survey of nature-inspired algorithms for feature selection to identify Parkinson’s disease. Computer Methods and Programs in Biomedicine, 139, 171–179.
https://doi.org/10.1016/j.cmpb.2016.07.029.
Taha, A. M., Mustapha, A., & Chen, S. Der. (2013). Naive Bayes-guided bat algorithm for feature selection. The Scientific World Journal, 2013. https://doi.org/10.1155/2013/325973.
Van Der Maaten, L. J. P., Postma, E. O., & Van Den Herik, H. J. (2009). Dimensionality Reduction: A Comparative Review. Journal of Machine Learning Research, 10, 1–41. https://doi.org/10.1080/13506280444000102.
Vergara, J. R., & Estévez, P. A. (2014). A review of feature selection methods based on mutual information. Neural Computing and Applications, 24(1), 175–186. https://doi.org/10.1007/s00521-013-1368-0.
Yang, J., & Honavar, V. (1998). Feature subset selection using genetic algorithm. IEEE Intelligent Systems and Their Applications, 13(2), 44–48. https://doi.org/10.1109/5254.671091.
Yang, X. S. (2013). Bat algorithm: Literature review and applications. International Journal of Bio-Inspired Computation, 5(3), 141–149. https://doi.org/10.1504/IJBIC.2013.055093.
Yu, L., & Liu, H. (2003). Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution. Proceedings, Twentieth International Conference on Machine Learning, 2, 856–863.
ZorarpacI, E., & Özel, S. A. (2016). A hybrid approach of differential evolution and artificial bee colony for feature selection. Expert Systems with Applications, 62, 91–103. https://doi.org/10.1016/j.eswa.2016.06.004.
Abbas, D. K. et al. (2022) ‘Using fitness dependent optimizer for training multi-layer perceptron’, arXiv preprint arXiv:2201.00563.
Abdulkhaleq, M. T. et al. (no date) ‘Fitness Dependent Optimizer with Neural Networks for COVID-19 Patients’. Abualigah, L. M., Khader, A. T. and Hanandeh, E. S. (2018) ‘A new feature selection method to improve the document clustering using particle swarm optimization algorithm’, Journal of Computational Science, 25, pp. 456–466.
Aghdam, M. H., Ghasem-Aghaee, N. and Basiri, M. E. (2009) ‘Text feature selection using ant colony optimization’, Expert systems with applications, 36(3), pp. 6843–6853.
Aghdam, M. H. and Heidari, S. (2015) ‘Feature selection using particle swarm optimization in text categorization’, Journal of Artificial Intelligence and Soft Computing Research, 5.
Aghdam, M. H., Kabiri, P. and others (2016) ‘Feature selection for intrusion detection system using ant colony optimization.’, Int. J. Netw. Secur., 18(3), pp. 420–432.
Ahdesmäki, M. and Strimmer, K. (2010) ‘Feature selection in omics prediction problems using cat scores and false nondiscovery rate control’, The Annals of Applied Statistics, 4(1), pp. 503–519.
Ahmad, I. (2015) ‘Feature selection using particle swarm optimization in intrusion detection’, International Journal of Distributed Sensor Networks.
Al-Ani, A. (2005) ‘Feature subset selection using ant colony optimization’, International journal of computational intelligence.
Al-Ani, A., Alsukker, A. and Khushaba, R. N. (2013) ‘Feature subset selection using differential evolution and a wheel based search strategy’, Swarm and Evolutionary Computation, 9, pp. 15–26.
Alarifi, A. et al. (2020) ‘A big data approach to sentiment analysis using greedy feature selection with cat swarm optimization-based long short-term memory neural networks’, The Journal of Supercomputing, 76(6), pp. 4414–4429.
Almarzouki, H. Z. (2022) ‘Deep-Learning-Based Cancer Profiles Classification Using Gene Expression Data Profile’, Journal of Healthcare Engineering, 2022.
Arslan, S. and Ozturk, C. (2019) ‘Multi hive artificial bee colony programming for high dimensional symbolic regression with feature selection’, Applied Soft Computing, 78, pp. 515–527.
Bansal, P. et al. (2020) ‘A hybrid grasshopper and new cat swarm optimization algorithm for feature selection and optimization of multi-layer perceptron’, Soft computing, 24(20), pp. 15463–15489.
Bhattacharyya, S. et al. (2014) ‘Automatic feature selection of motor imagery EEG signals using differential evolution and learning automata’, Medical & biological engineering & computing, 52(2), pp. 131–139.
Brezočnik, L. (2017) ‘Feature selection for classification using particle swarm optimization’, in IEEE EUROCON 2017-17th International Conference on Smart Technologies, pp. 966–971.
Bui, Q.-T. et al. (2019) ‘Whale Optimization Algorithm and Adaptive Neuro-Fuzzy Inference System: a hybrid method for feature selection and land pattern classification’, International journal of remote sensing, 40(13), pp. 5078–5093.
Chiu, P. C. et al. (2021) ‘Hybrid Sine Cosine and Fitness Dependent Optimizer for Global Optimization’, IEEE Access, 9, pp. 128601–128622.
Dadaneh, B. Z., Markid, H. Y. and Zakerolhosseini, A. (2016) ‘Unsupervised probabilistic feature selection using ant colony optimization’, Expert Systems with Applications, 53, pp. 27–42.
Deriche, M. (2009) ‘Feature selection using ant colony optimization’, in 2009 6th International Multi-Conference on Systems, Signals and Devices, pp. 1–4.
Dixit, A., Mani, A. and Bansal, R. (2020) ‘Feature selection for text and image data using differential evolution with SVM and Na{"i}ve Bayes classifiers’, Engineering Journal, 24(5), pp. 161–172.
Emary, E. et al. (2015) ‘Multi-objective gray-wolf optimization for attribute reduction’, Procedia Computer Science, 65, pp. 623–632.
Fallahzadeh, O., Dehghani-Bidgoli, Z. and Assarian, M. (2018) ‘Raman spectral feature selection using ant colony optimization for breast cancer diagnosis’, Lasers in medical science, 33(8), pp. 1799–1806.
Gao, X.-Z. et al. (2021) ‘Multi-objective optimization of feature selection using hybrid cat swarm optimization’, Science China Technological Sciences, 64(3), pp. 508–520.
Gomathy, M. (2021) ‘Optimal feature selection for speech emotion recognition using enhanced cat swarm optimization algorithm’, International Journal of Speech Technology, 24(1), pp. 155–163.
Guha, R. et al. (2020) ‘Embedded chaotic whale survival algorithm for filter--wrapper feature selection’, Soft Computing, 24(17), pp. 12821–12843.
Hancer, E. et al. (2018) ‘Pareto front feature selection based on artificial bee colony optimization’, Information Sciences, 422, pp. 462–479.
Hancer, E. (2020) ‘New filter approaches for feature selection using differential evolution and fuzzy rough set theory’, Neural Computing and Applications, 32(7), pp. 2929–2944.
Hancer, E., Xue, B. and Zhang, M. (2022) ‘Fuzzy filter cost-sensitive feature selection with differential evolution’, Knowledge-Based Systems, 241, p. 108259.
Jayaprakash, A. and KeziSelvaVijila, C. (2019) ‘Feature selection using ant colony optimization (ACO) and road sign detection and recognition (RSDR) system’, Cognitive Systems Research, 58, pp. 123–133.
Jeyasingh, S. and Veluchamy, M. (2017) ‘Modified bat algorithm for feature selection with the wisconsin diagnosis breast cancer (WDBC) dataset’, Asian Pacific journal of cancer prevention: APJCP, 18(5), p. 1257.
Johari, P. K. and Gupta, R. K. (2021) ‘Improved Feature Selection Techniques for Image Retrieval based on Metaheuristic Optimization’, International Journal of Computer Science & Network Security, 21(1), pp. 40–48.
Kabir, M. et al. (2009) ‘An efficient feature selection using ant colony optimization algorithm’, in International Conference on Neural Information Processing, pp. 242–252.
Kanan, H. R., Faez, K. and Taheri, S. M. (2007) ‘Feature selection using ant colony optimization (ACO): a new method and comparative study in the application of face recognition system’, in Industrial conference on data mining, pp. 63–76.
Khushaba, R. N., Al-Ani, A. and Al-Jumaily, A. (2008) ‘Feature subset selection using differential evolution’, in International Conference on Neural Information Processing, pp. 103–110.
Khushaba, R. N., Al-Ani, A. and Al-Jumaily, A. (2011) ‘Feature subset selection using differential evolution and a statistical repair mechanism’, Expert Systems with Applications, 38(9), pp. 11515–11526.
Kitonyi, P. M. and Segera, D. R. (2021) ‘Hybrid gradient descent grey wolf optimizer for optimal feature selection’, BioMed Research International, 2021.
Kumar Gupta, D. et al. (2015) ‘Pso-asent: Feature selection using particle swarm optimization for aspect based sentiment analysis’, in International conference on applications of natural language to information systems, pp. 220–233.
Li, Q. et al. (2017) ‘An enhanced grey wolf optimization based feature selection wrapped kernel extreme learning machine for medical diagnosis’, Computational and mathematical methods in medicine, 2017.
Lin, K.-C. et al. (2015) ‘Feature selection and parameter optimization of support vector machines based on modified cat swarm optimization’, International Journal of Distributed Sensor Networks, 11(7), p. 365869.
Lin, K.-C. et al. (2016) ‘Feature selection based on an improved cat swarm optimization algorithm for big data classification’, The Journal of Supercomputing, 72(8), pp. 3210–3221.
Lin, K. C., Zhang, K. Y. and Hung, J. C. (2014) ‘Feature selection of support vector machine based on harmonious cat swarm optimization’, in 2014 7th international conference on ubi-media computing and workshops, pp. 205–208.
Liu, Y. et al. (2011) ‘An improved particle swarm optimization for feature selection’, Journal of Bionic Engineering, 8(2), pp. 191–200.
Mandal, M. et al. (2021) ‘A tri-stage wrapper-filter feature selection framework for disease classification’, Sensors, 21(16), p. 5571.
Moradi, P. and Rostami, M. (2015) ‘Integration of graph clustering with ant colony optimization for feature selection’, Knowledge-Based Systems, 84, pp. 144–161.
Nakamura, R. Y. M. et al. (2012) ‘BBA: a binary bat algorithm for feature selection’, in 2012 25th SIBGRAPI conference on graphics, patterns and images, pp. 291–297.
Nematzadeh, H. et al. (2019) ‘Frequency based feature selection method using whale algorithm’, Genomics, 111(6), pp. 1946–1955.
Nirmala Sreedharan, N. P. et al. (2018) ‘Grey wolf optimisation-based feature selection and classification for facial emotion recognition’, IET Biometrics, 7(5), pp. 490–499.
Qasim, O. S. and Algamal, Z. Y. (2018) ‘Feature selection using particle swarm optimization-based logistic regression model’, Chemometrics and Intelligent Laboratory Systems, 182, pp. 41–46.
Sahu, B. and Mishra, D. (2012) ‘A novel feature selection algorithm using particle swarm optimization for cancer microarray data’, Procedia Engineering, 38, pp. 27–31.
Saleem, N., Zafar, K. and Sabzwari, A. F. (2019) ‘Enhanced feature subset selection using Niche based bat algorithm’, Computation, 7(3), p. 49.
Salih, J. F., Mohammed, H. M. and Abdul, Z. K. (2022) ‘Modified Fitness Dependent Optimizer for Solving Numerical Optimization Functions’, IEEE Access, 10, pp. 83916–83930. doi: 10.1109/ACCESS.2022.3197290.
Tawhid, M. A. and Dsouza, K. B. (2018) ‘Hybrid binary bat enhanced particle swarm optimization algorithm for solving feature selection problems’, Applied Computing and Informatics.
Too, J. et al. (2018) ‘A new competitive binary grey wolf optimizer to solve the feature selection problem in EMG signals classification’, Computers, 7(4), p. 58.
Too, J., Mafarja, M. and Mirjalili, S. (2021) ‘Spatial bound whale optimization algorithm: an efficient high-dimensional feature selection approach’, Neural Computing and Applications, 33(23), pp. 16229–16250.
Vivekanandan, T. and Iyengar, N. C. S. N. (2017) ‘Optimal feature selection using a modified differential evolution algorithm and its effectiveness for prediction of heart disease’, Computers in biology and medicine, 90, pp. 125–136.
Wang, X. et al. (2020) ‘Multi-objective feature selection based on artificial bee colony: An acceleration approach with variable sample size’, Applied Soft Computing, 88, p. 106041.
Xue, B., Zhang, M. and Browne, W. N. (2012) ‘Particle swarm optimization for feature selection in classification: A multi-objective approach’, IEEE transactions on cybernetics, 43(6), pp. 1656–1671.
Yang, X. S. (2013) ‘Na{"i}ve Bayes Guided Bat algorithm for featureselection’, TheScientificWorldJournalVolume20, 13.
Zahran, B. M. and Kanaan, G. (2009) ‘Text feature selection using particle swarm optimization algorithm 1’.
Zhang, Y. et al. (2019) ‘Cost-sensitive feature selection using two-archive multi-objective artificial bee colony algorithm’, Expert Systems with Applications, 137, pp. 46–58.
Zhang, Y. et al. (2020) ‘Binary differential evolution with self-learning for multi-objective feature selection’, Information Sciences, 507, pp. 67–85.
Zheng, Y. et al. (2018) ‘A novel hybrid algorithm for feature selection based on whale optimization algorithm’, IEEE Access, 7, pp. 14908–14923.
Zorarpaci, E. and Özel, S. A. (2016) ‘A hybrid approach of differential evolution and artificial bee colony for feature selection’, Expert Systems with Applications, 62, pp. 91–103.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Zana O. Hamad
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-ND 4.0] that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Polytechnic Journal ; A Periodical Open Access Scientific Journal