Improved Sketch-to-Photo Generation Using Filter Aided Generative Adversarial Network
Article Sidebar
Main Article Content
Abstract
Generating a photographic face image from given input sketch is most challenging task in computer vision. Mainly the sketches drawn by sketch artist used in human identification. Sketch to photo synthesis is very important applications in law enforcement as well as character design, educational training. In recent years Generative Adversarial Network (GAN) shows excellent performance on sketch to photo synthesis problem. Quality of hand drawn sketches affects the quality generated photo. It might be possible that while handling the hand drawn sketches, accidently by touching the user hand on pencil sketch or similar activities causes noise in given sketch. Likewise different styles like shading, darkness of pencil used by sketch artist may cause unnecessary noise in sketches. In recent year many sketches to photo synthesis methods are proposed, but they are mainly focused on network architecture to get better performance. In this paper we proposed Filter-aided GAN framework to remove such noise while synthesizing photo images from hand drawn sketches. Here we implement and compare different filtering methods with GAN. Quantitative and qualitative result shows that proposed Filter-aided GAN generate the photo images which are visually pleasant and closer to ground truth image.
Article Details
References
N. Wang, M. Zhu, J. Li, B. Song, and Z. Li, “Data-driven vs. modeldriven: Fast face sketch synthesis,” Neurocomputing, vol. 257, pp. 214– 221, 2017
X. Wang and X. Tang, “Face photo-sketch synthesis and recognition”, In: IEEE Transactions on Pattern Analysis and Machine Intelligence”, 2009
X. Gao, N. Wang, D. Tao, and X. Li, “Face sketch photo synthesis and retrieval using sparse representation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 8, pp. 1213–1226, 2012.
T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu. “Sketch2photo: internet image montage” In: ACM Transactions on Graphics, 2009.
Y. Song, L. Bao, Q. Yang, and M. H. Yang, “Real-time exemplar-based face sketch synthesis,” in European Conference on Computer Vision, 2014, pp. 800–813.
Y. Song, L. Bao, S. He, Q. Yang, and M.-H. Yang, “Stylizing face images via multiple exemplars,” Computer Vision and Image Understanding, vol. 162, pp. 135–145, 2017.
S. Zhang, X. Gao, N. Wang, J. Li, and M. Zhang, “Face sketch synthesis via sparse representation-based greedy search.” IEEE Transactions on Image Processing, vol. 24, no. 8, pp. 2466–77, 2015
Ahmed Cherif Megri, Sameer Hamoush, Ismail Zayd Megri, Yao Yu. (2021). Advanced Manufacturing Online STEM Education Pipeline for Early-College and High School Students. Journal of Online Engineering Education, 12(2), 01–06. Retrieved from http://onlineengineeringeducation.com/index.php/joee/article/view/47
N. Wang, D. Tao, X. Gao, X. Li, and J. Li, “Transductive face sketchphoto synthesis,” IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 9, pp. 1364–1376, 2013.
L. Zhang, L. Lin, X. Wu, S. Ding, L. Zhang, End-to-end photo-sketch generation via fully convolutional representation learning, in: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, in: ICMR ’15, ACM, New York, NY, USA, 2015, pp. 627–634.
D. Zhang, L. Lin, T. Chen, X. Wu, W. Tan, and E. Izquierdo, “Contentadaptive sketch portrait generation by decompositional representation learning,” IEEE Transactions on Image Processing, vol. 26, no. 1, pp. 328–339, 2016.
Monowar, M. M. (2022). TRW-MAC: A thermal-aware receiver-driven wake-up radio enabled duty cycle MAC protocol for multi-hop implantable wireless body area networks in Internet of Things. International Journal of Communication Networks and Information Security (IJCNIS), 14(1).
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, DavidWarde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates, Inc., 2014.
Lu Y., Wu S., Tai YW., Tang CK. (2018) Image Generation from Sketch Constraint Using Contextual GAN. In: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11220. Springer, Cham.
Mohammed, M. M., Viktor, Z., Yurii, K., Lyubov, B., Konstiantyn, P., & Turovsky, O. L. (2022). Methods for assessing the impact of bandwidth of control channels on the quality of telecommunications networks in the transmission of data packets of different types. International Journal of Communication Networks and Information Security (IJCNIS), 13(2).
J. Zhao, X. Xie, L. Wang, M. Cao and M. Zhang, "Generating Photographic Faces from the Sketch Guided by Attribute Using GAN," in IEEE Access, vol. 7, pp. 23844-23851, 2019
Uche Osahor, Hadi Kazemi, Ali Dabouei, Nasser Nasrabadi,(2020), “Quality Guided Sketch-to-Photo Image Synthesis”, In:Computer Vision And Pattern Recognition (CVPR).
Yu et al., "Toward Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs," in IEEE Transactions on Cybernetics, doi: 10.1109/TCYB.2020
Chao, Wentao et al. “High-Fidelity Face Sketch-To-Photo Synthesis Using Generative Adversarial Network,” 2019 IEEE International Conference on Image Processing (ICIP) (2019): 4699-4703
Lidan Wang et al. “High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks”, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition
Wang, V. Sindagi and V. Patel, "High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks," 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi'an, 2018, pp. 83-90.
Hadi Kazemi et al. “Facial Attributes Guided Deep Sketch-to-Photo Synthesis”,2018 IEEE Winter Conference on Applications of Computer Vision Workshops
Joy, P., Thanka, R., & Edwin, B. (2022). Smart Self-Pollination for Future Agricultural-A Computational Structure for Micro Air Vehicles with Man-Made and Artificial Intelligence. International Journal of Intelligent Systems and Applications in Engineering, 10(2), 170–174. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/1743
Yuke Fang Yuke Fang “Identity-aware CycleGAN for face photo-sketch synthesis and recognition,” Pattern Recognition,Volume 102,2020,107249,ISSN 0031-3203.
Xiaogang Wang and Xiaoou Tang, “Face photo-sketch synthesis and recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009.
D. P. Kingma and J. Ba. (2014). ‘‘Adam: A method for stochastic optimization.’’ [Online]. Available: https://arxiv.org/abs/1412.6980
Kumar, S., Gornale, S. S., Siddalingappa, R., & Mane, A. (2022). Gender Classification Based on Online Signature Features using Machine Learning Techniques. International Journal of Intelligent Systems and Applications in Engineering, 10(2), 260–268. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/2020
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600–612, 2004.
S. L. Bangare, “Classification of optimal brain tissue using dynamic region growing and fuzzy min-max neural network in brain magnetic resonance images”, Neuroscience Informatics, Volume 2, Issue 3, September 2022, 100019, ISSN 2772-5286, https://doi.org/10.1016/j.neuri.2021.100019 .
S. L. Bangare, G. Pradeepini, S. T. Patil, “Implementation for brain tumor detection and three dimensional visualization model development for reconstruction”, ARPN Journal of Engineering and Applied Sciences (ARPN JEAS), Vol.13, Issue.2, ISSN 1819-6608, pp.467-473. 20/1/2018 http://www.arpnjournals.org/jeas/research_papers/rp_2018/jeas_0118_6691.pdf
Sandeep Pande and Manna Sheela Rani Chetty, “Bezier Curve Based Medicinal Leaf Classification using Capsule Network”, International Journal of Advanced Trends in Computer Science and Engineering, Vol. 8, No. 6, pp. 2735-2742, 2019.
Pande S.D., Chetty M.S.R. (2021) Fast Medicinal Leaf Retrieval Using CapsNet. In: Bhattacharyya S., Nayak J., Prakash K.B., Naik B., Abraham A. (eds) International Conference on Intelligent and Smart Computing in Data Analytics. Advances in Intelligent Systems and Computing, vol 1312.
P. S. Bangare, S. L. Bangare, R. U. Yawle and S. T. Patil, "Detection of human feature in abandoned object with modern security alert system using Android Application," 2017 International Conference on Emerging Trends & Innovation in ICT (ICEI), 2017, pp. 139-144, doi: 10.1109/ETIICT.2017.7977025
N. Shelke, S. Chaudhury, S. Chakrabarti, S. L. Bangare et al. “An efficient way of text-based emotion analysis from social media using LRA-DNN”, Neuroscience Informatics, Volume 2, Issue 3, September 2022, 100048, ISSN 2772-5286, https://doi.org/10.1016/j.neuri.2022.100048 .
Suneet Gupta, Sumit Kumar, Sunil L. Bangare, Shibili Nuhmani, Arnold C. Alguno, Issah Abubakari Samori, “Homogeneous Decision Community Extraction Based on End-User Mental Behavior on Social Media”, Computational Intelligence and Neuroscience, vol. 2022, Article ID 3490860, 9 pages, 2022. https://doi.org/10.1155/2022/3490860.
Gururaj Awate, S. L. Bangare, G. Pradeepini and S. T. Patil, “Detection of Alzheimers Disease from MRI using Convolutional Neural Network with Tensorflow”,arXiv, https://doi.org/10.48550/arXiv.1806.10170