A Novel Approach for Optimization of Convolution Neural Network with Particle Swarm Optimization and Genetic Algorithm for Face Recognition

Article Sidebar

PDF
Published: May 5, 2023
DOI: https://doi.org/10.17762/ijritcc.v11i4s.6531
Keywords:
CNN, Hyperparameter Optimization, Genetic algorithm, Particle Swarm Optimization, Face recognition

Main Article Content

Kalaiarasi P
Research Scholar, Department of Electronics and Communication Engineering Vel Tech Rangarajan Dr.Sagunthala R&D Institute of Science and Technology Avadi,Chennai 600062, India
P. Esther Rani
Professor, Department of Electronics and Communication Engineering Vel Tech Rangarajan Dr.Sagunthala R&D Institute of Science and Technology Avadi, Chennai 600062, India

Abstract

Convolutional neural networks are contemporary deep learning models that are employed for many various applications. In general, the filter size, number of filters, number of convolutional layers, number of fully connected layers, activation function and learning rate are some of the hyperparameters that significantly determine how well a CNN performs.. Generally, these hyperparameters are selected manually and varied for each CNN model depending on the application and dataset. During optimization, CNN could get stuck in local minima. To overcome this, metaheuristic algorithms are used for optimization. In this work, the CNN structure is first constructed with randomly chosen hyperparameters and these parameters are optimized using Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) algorithm. A CNN with optimized hyperparameters is used for face recognition. CNNs optimized with these algorithms use RMSprop optimizer instead of stochastic gradient descent. This RMSprop optimizer helps the CNN reach global minimum quickly. It has been observed that optimizing with GA and PSO improves the performance of CNNs. It also reduces the time it takes for the CNN to reach the global minimum.

Article Details

How to Cite
P, K. ., & Rani, P. E. . (2023). A Novel Approach for Optimization of Convolution Neural Network with Particle Swarm Optimization and Genetic Algorithm for Face Recognition. International Journal on Recent and Innovation Trends in Computing and Communication, 11(4s), 215–223. https://doi.org/10.17762/ijritcc.v11i4s.6531
Issue
Vol. 11 No. 4s (2023)
Section
Articles

References

V. Vijayakumari, “Face Recognition Techniques: A Survey,” World Journal of Computer Application and Technology, 1(2), 41-50, 2013.

A. V. Nefian and M. H. Hayes, “Hidden markov models for face detection and recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 1, 141-145, 1999.

G. Carvajal, W. Valenzuela and M. Figueroa, “Subspace-based face recognition in analog VLSl,” Proceedings of the 20th Neural Information Processing Systems, 225-232, Vacouver, 2007.

L. Juwei, K. N. Plataniotis, L. D. Hannon, M. K. lilian, M. R. Lasch and P. F. Ramig, “Machine identification of human faces,” Pattern Recognition, 13(2), 97-110, 1981.

A. Krizhevsky, I. Sutskever and G.E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, 1097-1105, 2012.

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al, “Going deeper with convolutions,” Proceedings CVPR IEEE, 1–9, 2015.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” Proceedings International Conference on Learning Representations, 2015.

K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” Proceedings CVPR IEEE, 2016.

F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally and K. Keutzer,” SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5MB model size,” Proceedings CVPR IEEE, arXiv:1602.07360 [cs.CV], 2016.

Y. LeCun and Y. Bengio, “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, 3361(10). 1995.

J. H. Holland, “Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence,” MIT press, 1992.

P. Esfahanian and M. Akhavan M, “GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm,” Neural and Evolutionary Computing. arXiv: 1909.13354, 2019.

Mei-Ping Song and Guo-Chang Gu “Research on particle swarm optimization: a review”. Proceedings of International Conference on Machine Learning and Cybernetics, Vol.4. (IEEE Cat.No.04EX826). pp. 2236- 2241. Shanghai. China. DOI: 10.1109/ICMLC.2004.1382171, 2004.

Y. Sun, B. Xue, M. Zhang, G. G. Yen, J. Lv, “Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification,” IEEE Transactions on Cybernetics, 50(9), pp.3840-3854. DOI: 10.1109/TCYB.2020.2983860, 2020.

N. M. Aszemi, and P. D. D. Dominic, “Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms,” International Journal of Advanced Computer Science and Applications. 10(6). DOI:10.14569/IJACSA.2019.0100638, 2019.

B. Wang, Y. Sun, B. Xue and M. Zhang, “A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks,” Proceedings CVPR IEEE, 2019.

M. A. Turk and A. P. Pentland, “Face recognition using eigen faces,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Maui, HI, USA, pp. 586-591, doi: 10.1109/CVPR.1991.139758, 1991.

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Transactions on Pattern Analysis and Machine Intelligence. 19(7). pp. 711-720. DOI: 10.1109/34.598228, 1997.

W. Ge, W. Quan and C. Han, “Face description and identification using histogram sequence of local binary pattern,” Seventh International Conference on Advanced Computational Intelligence (ICACI, Wuyi, China. pp. 415-420, DOI: 10.1109/ICACI.2015.7184743, 2015.

F. Purnomo, D. Suhartono, M. Shodiq, A. Susanto, S. Raharja and W. Kurniawan, “Face recognition using Gabor Wavelet and Non-negative Matrix Factorization,” SAI Intelligent Systems Conference (IntelliSys), London, UK, pp. 788-792. DOI: 10.1109/IntelliSys.2015.7361230, 2015.

F. Radzi, M. Khalil-Hani, S. S. Liew and R. Bakhteri, “Convolutional Neural Network for Face Recognition with Pose and Illumination Variation,” International Journal of Engineering and Technology, 6(1), 2014.

T. Goel and R. Murugan, R, “Classifier for Face Recognition Based on Deep Convolutional – Optimized Kernel Extreme Learning Machine,” Computers & Electrical Engineering, Vol.85, 2020, DOI:10.1016/j.compeleceng.2020.106640. 2020.

F. Erivaldo, F. Junior and G. G. Yen, “Particle swarm optimization of deep neural networks architectures for image classification. Swarm and Evolutionary Computation, vol.49, 2019, DOI: 10.1016/j.swevo.2019.05.010. 2019. pp 62-74.

F. Ye,”Particle swarm optimization-based automatic parameter selection for deep neural networks and its applications in large-scale and high-dimensional data,” PLoS ONE, 12(12), 2012.

Y. Sun, B. Xue, M. Zhang and G. G. Yen, “A Particle Swarm Optimization-Based Flexible Convolutional Autoencoder for Image Classification,” IEEE Transactions on Neural Networks and Learning Systems, 30(8), pp. 2295-2309. DOI: 10.1109/TNNLS.2018.2881143, 2019.

J. Fregoso, C. I. Gonzalez and G. E. Martinez, “Optimization of Convolutional Neural Networks Architectures Using PSO for Sign Language Recognition”. Axioms, 10(3), 139, 2021, DOI: 10.3390/axioms10030139.

J. R. Challapalli and N. Devarakonda, “A novel approach for optimization of convolution neural network with hybrid particle swarm and grey wolf algorithm for classification of Indian classical dances,” Knowledge and Information System, 64:2411–2434. DOI: 10.1007/s10115-022-01707-3, 2022.

N. M. Aszemi and P. D. D. Dominic, “Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms,” International Journal of Advanced Computer Science and Applications, 10(6). DOI:10.14569/IJACSA.2019.0100638, 2019.

E. Russell and J. Kennedy, “A new optimizer using particle swarm theory,” Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan (4-6 Oct. 1995). pp: 39-43. DOI:10.1109/MHS.1995.494215.

J. Kennedy and R. C. Eberhart, “Particle Swarm Optimization,” Proceedings of IEEE International Conference on Neural Networks, Perth. Australia. Pp. 1942-1948, 1995.

Faces94 dataset, https://cswww.essex.ac.uk/mv/allfaces/faces94.html.

Yale dataset, http://vision.ucsd.edu/datasets/yale_face_dataset_original/yalefaces.zip.

ORL dataset, https://www.face-rec.org/databases/

S. A. Padmanabhan and J. Kanchikere, “An efficient face recognition system based on hybrid optimized KELM,” Multimedia Tools Application, 79, pp: 10677–10697, 2020.

W. Yulin and J. Mingvan, “Face recognition system based on CNN and LBP features for classifier optimization and fusion,” The Journal of China Universities of Posts and Telecommunications. 25(1):37-47, 2018.

P. G. Devarakonda and B. Bozic, “Particle Swarm Optimization of Convolutional Neural Network for Human Activity Prediction,” Optimization Algorithms of Swarm Intelligence, 2022, DOI: 10.5772/intechopen.97259.