An automatic approach for classification and categorisation of lip morphological traits

English version

Autoři: Hawraa H. Abbas ^aff001; Yulia Hicks ^aff002; Alexei Zhurov ^aff004; David Marshall ^aff003; Peter Claes ^aff005; Caryl Wilson-Nagrani ^aff004; Stephen Richmond ^aff004
Působiště autorů: School of Engineering, Kerbala University, Kerbala, Iraq ^aff001; School of Engineering, Cardiff University, Cardiff, Wales, United Kingdom ^aff002; School of Computer Science and Informatics, Cardiff University, Cardiff, Wales, United Kingdom ^aff003; School of Dentistry, Cardiff University, Cardiff, Wales, United Kingdom ^aff004; Medical Imaging Research Center, University of Leuven, Leuven, Belgium ^aff005
Vyšlo v časopise: PLoS ONE 14(10)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0221197

Souhrn

Classification of facial traits (e.g., lip shape) is an important area of medical research, for example, in determining associations between lip traits and genetic variants which may lead to a cleft lip. In clinical situations, classification of facial traits is usually performed subjectively directly on the individual or recorded later from a three-dimensional image, which is time consuming and prone to operator errors. The present study proposes, for the first time, an automatic approach for the classification and categorisation of lip area traits. Our approach uses novel three-dimensional geometric features based on surface curvatures measured along geodesic paths between anthropometric landmarks. Different combinations of geodesic features are analysed and compared. The effect of automatically identified categories on the face is visualised using a partial least squares method. The method was applied to the classification and categorisation of six lip shape traits (philtrum, Cupid’s bow, lip contours, lip-chin, and lower lip tone) in a large sample of 4747 faces of normal British Western European descents. The proposed method demonstrates correct automatic classification rate of up to 90%.

Klíčová slova:

Algorithms – Analysis of variance – Face recognition – Face – Anthropometry – Lips – Geodesics – Curvature

Zdroje

1. Solow B, Tallgren A. Natural head position in standing subjects. Acta Odontologica Scandinavica. 1971;29(5):591–607. doi: 10.3109/00016357109026337 5290983

2. Sforza C, Grandi G, Binelli M, Dolci C, De Menezes M, Ferrario VF. Age- and sex-related changes in three-dimensional lip morphology. Forensic science international. 2010;200(1):182–e1. doi: 10.1016/j.forsciint.2010.04.050 20570070

3. De Menezes M, Rosati R, Baga I, Mapelli A, Sforza C. Three-dimensional analysis of labial morphology: effect of sex and age. International journal of oral and maxillofacial surgery. 2011;40(8):856–861. doi: 10.1016/j.ijom.2011.03.004 21477995

4. Shelke RG, Annadate S. Face Recognition and Gender Classification Using Feature of Lips. International Journal of Innovation and Scientific Research (IJISR), Innovative Space of Scientific Research Journals (ISSR). 2014;10(2).

5. Stewart D, Pass A, Zhang J. Gender classification via lips: static and dynamic features. IET biometrics. 2013;2(1):28–34.

6. Koller O, Ney H, Bowden R. Deep Learning of Mouth Shapes for Sign Language. In: Proceedings of the IEEE International Conference on Computer Vision Workshops; 2015. p. 85–91.

7. Saitoh T, Hisagi M, Konishi R. Japanese Phone Recognition Using Lip Image Information. In: MVA. Citeseer; 2007. p. 134–137.

8. Holdaway RA. A soft-tissue cephalometric analysis and its use in orthodontic treatment planning. Part I. American journal of orthodontics. 1983;84(1):1–28. doi: 10.1016/0002-9416(83)90144-6 6575614

9. Merrifield LL. The profile line as an aid in critically evaluating facial esthetics. American journal of orthodontics. 1966;52(11):804–822. doi: 10.1016/0002-9416(66)90250-8 5223046

10. Ricketts RM. Esthetics, environment, and the law of lip relation. American journal of orthodontics. 1968;54(4):272–289. doi: 10.1016/s0002-9416(68)90278-9 5238879

11. Burstone CJ. Lip posture and its significance in treatment planning. American journal of orthodontics. 1967;53(4):262–284. doi: 10.1016/0002-9416(67)90022-x 5227460

12. Wang KH, Heike CL, Clarkson MD, Mejino JL, Brinkley JF, Tse RW, et al. Evaluation and integration of disparate classification systems for clefts of the lip. Frontiers in physiology. 2014;5:163. doi: 10.3389/fphys.2014.00163 24860508

13. Beaty T, Taub M, Scott A, Murray J, Marazita M, Schwender H, et al. Confirming genes influencing risk to cleft lip with/without cleft palate in a case–parent trio study. Human genetics. 2013;132(7):771–781. doi: 10.1007/s00439-013-1283-6

14. Suttie M, Foroud T, Wetherill L, Jacobson JL, Molteno CD, Meintjes EM, et al. Facial dysmorphism across the fetal alcohol spectrum. Pediatrics. 2013;131(3):e779–e788. doi: 10.1542/peds.2012-1371 23439907

15. Abbas H, Hicks Y, Marshall D, Zhurov A, Richmond S. A 3D morphometric perspective for facial gender analysis and classification using geodesic path curvature features. Computational Visual Media. 2017.

16. Wilson C, Playle R, Toma A, Zhurov A, Ness A, Richmond S. The prevalence of lip vermilion morphological traits in a 15-year-old population. American Journal of Medical Genetics Part A. 2013;161(1):4–12.

17. Hwang K, Kim DJ, Hwang SH. Musculature of the pars marginalis of the upper orbicularis oris muscle. Journal of Craniofacial Surgery. 2007;18(1):151–154. doi: 10.1097/01.scs.0000248649.77168.ec 17251855

18. Heidari Z, Mahmoudzadeh-Sagheb H, Rad AA, Dahmardeh N. Anthropometric measurements of the lips in 18–25-year-old men of Sistani and Baluch descent. Bull Env Pharmacol Life Sci. 2014;3:139–142.

19. Carey JC, Cohen MM, Curry CJ, Devriendt K, Holmes LB, Verloes A. Elements of morphology: standard terminology for the lips, mouth, and oral region. American Journal of Medical Genetics Part A. 2009;149(1):77–92.

20. Shi JY, Zhou H, Mao RY, Chen Y, Li JT, Huo HY. A preliminary study on the key factors contributing to the attractive lips of Chinese children. Asian Pacific journal of tropical medicine. 2012;5(4):318–322. doi: 10.1016/S1995-7645(12)60047-9 22449526

21. O’Toole AJ, Vetter T, Troje NF, Bülthoff HH. Sex classification is better with three-dimensional head structure than with image intensity information. Perception. 1997;26(1):75–84. doi: 10.1068/p260075 9196691

22. Bruce V, Young A. Understanding face recognition. British journal of psychology. 1986;77(3):305–327. doi: 10.1111/j.2044-8295.1986.tb02199.x 3756376

23. Mori A, Nakajima T, Kaneko T, Sakuma H, Aoki Y. Analysis of 109 Japanese children’s lip and nose shapes using 3-dimensional digitizer. British journal of plastic surgery. 2005;58(3):318–329. doi: 10.1016/j.bjps.2004.11.019

24. Lee J, Ku B, Combs PD, Da Silveira AC, Markey MK. Quantitative Anthropometric Measures of Facial Appearance of Healthy Hispanic/Latino White Children: Establishing Reference Data for Care of Cleft Lip With or Without Cleft Palate. 3D Research. 2017;8(2):19.

25. Alpaydin E. Introduction to machine learning. MIT press; 2014.

26. Caruana R, Niculescu-Mizil A. An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on Machine learning. ACM; 2006. p. 161–168.

27. Enver K, Erdo P, Polat K. Brain MRI segmentation based on different clustering algorithms. International Journal of Computer Applications. 2016;155(3):37–40.

28. Anthony G, Gregg H, Tshilidzi M. Image classification using SVMs: one-against-one vs one-against-all. arXiv preprint arXiv:07112914. 2007.

29. Yuan X, Abouelenien M. A multi-class boosting method for learning from imbalanced data. International Journal of Granular Computing, Rough Sets and Intelligent Systems. 2015;4(1):13–29.

30. Arthur D, Vassilvitskii S. k-means++: The advantages of careful seeding. In: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms. Society for Industrial and Applied Mathematics; 2007. p. 1027–1035.

31. Toma AM. Characterization of normal facial features and their association with genes [PhD Thesis]. Cardiff University; 2014.

32. Wilamowska K, Wu J, Heike C, Shapiro L. Shape-based classification of 3D facial data to support 22q11.2DS craniofacial research. Journal of digital imaging. 2012;25(3):400–408. doi: 10.1007/s10278-011-9430-x 22086243

33. Abbas H, Hicks Y, Marshall D. Automatic classification of facial morphology for medical applications. Procedia Computer Science. 2015;60:1649–1658.

34. Perakis P, Theoharis T, Passalis G, Kakadiaris IA. Automatic 3D Facial Region Retrieval from Multi-pose Facial Datasets. In: 3DOR; 2009. p. 37–44.

35. Lu X, Jain AK. Automatic feature extraction for multiview 3D face recognition. In: Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on. IEEE; 2006. p. 585–590.

36. Lu X, Jain AK, Colbry D. Matching 2.5D face scans to 3D models. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2006;28(1):31–43.

37. Zhao J, Liu C, Wu Z, Duan F, Zhang M, Wang K, et al. 3D facial similarity measure based on geodesic network and curvatures. Mathematical Problems in Engineering. 2014;2014.

38. Ballihi L, Amor BB, Daoudi M, Srivastava A, Aboutajdine D. Boosting 3-D-geometric features for efficient face recognition and gender classification. IEEE Transactions on Information Forensics and Security. 2012;7(6):1766–1779.

39. Jahanbin S, Choi H, Liu Y, Bovik AC. Three dimensional face recognition using iso-geodesic and iso-depth curves. In: Biometrics: Theory, Applications and Systems, 2008. BTAS 2008. 2nd IEEE International Conference on. IEEE; 2008. p. 1–6.

40. Ahdid R, Barrah EM, Safi S, Manaut B. Facial Surface Analysis using Iso-Geodesic Curves in Three Dimensional Face Recognition System. arXiv preprint arXiv:160808878. 2016.

41. com S. Partial Least Squares (PLS);. Available from: http://www.statsoft.com/Textbook/Partial-Least-Squares.

42. Shrimpton S, Daniels K, De Greef S, Tilotta F, Willems G, Vandermeulen D, et al. A spatially-dense regression study of facial form and tissue depth: towards an interactive tool for craniofacial reconstruction. Forensic science international. 2014;234:103–110. doi: 10.1016/j.forsciint.2013.10.021 24378309

43. Claes P, Daniels K, Vandermeulen D, Suetens P, Shriver MD. A PLS regression framework for spatially-dense geometric morphometrics to analyze effects on shape and shape characteristics: applied to the study of genomic ancestry and sex on facial morphology. In: BIOLOGICAL SHAPE ANALYSIS: Proceedings of the 3rd International Symposium; 2015. p. 205–215.

44. Matthews H, Penington T, Saey I, Halliday J, Muggli E, Claes P. Spatially dense morphometrics of craniofacial sexual dimorphism in 1-year-olds. Journal of Anatomy. 2016;229(4):549–559. doi: 10.1111/joa.12507 27338586

45. Boyd A, Golding J, Macleod J, Lawlor DA, Fraser A, Henderson J, et al. Cohort profile: the ‘children of the 90s’—the index offspring of the Avon Longitudinal Study of Parents and Children. International journal of epidemiology. 2013;42(1):111–127. doi: 10.1093/ije/dys064

46. Kau CH, Richmond S. Three-dimensional analysis of facial morphology surface changes in untreated children from 12 to 14 years of age. American Journal of Orthodontics and Dentofacial Orthopedics. 2008;134(6):751–760. doi: 10.1016/j.ajodo.2007.01.037 19061801

47. Toma AM, Zhurov A, Playle R, Ong E, Richmond S. Reproducibility of facial soft tissue landmarks on 3D laser-scanned facial images. Orthodontics & craniofacial research. 2009;12(1):33–42.

48. Ruiz M, Illingworth J. Automatic landmarking of faces in 3D-ALF3D. In: Visual Information Engineering, 2008. VIE 2008. 5th International Conference on. IET; 2008. p. 41–46.

49. Koppen WP. Learning 3D face shape features from local coherence [PhD Thesis]. Centre for Vision, Speech and Signal Processing Faculty of Electronic Engineering University of Surrey; 2014.

50. de Jong MA, Wollstein A, Ruff C, Dunaway D, Hysi P, Spector T, et al. An automatic 3D facial landmarking algorithm using 2D Gabor wavelets. IEEE Transactions on Image Processing. 2016;25(2):580–588. doi: 10.1109/TIP.2015.2496183 26540684

51. Zhurov A, Richmond S, Kau CH, Toma A. Averaging facial images. In: Kau CH, Richmond S, editors. Three-dimensional imaging for orthodontics and maxillofacial surgery. Wiley-Blackwell; 2010. p. 126–144.

52. Hammond P, Suttie M, Hennekam RC, Allanson J, Shore EM, Kaplan FS. The face signature of fibrodysplasia ossificans progressiva. American Journal of Medical Genetics Part A. 2012;158(6):1368–1380.

53. Claes P, Walters M, Clement J. Improved facial outcome assessment using a 3D anthropometric mask. International journal of oral and maxillofacial surgery. 2012;41(3):324–330. doi: 10.1016/j.ijom.2011.10.019 22103995

54. Zhurov AI, Kau CH, Richmond S. Computer methods for measuring 3D facial morphology. In: Middleton J, Shrive N, Jones M, editors. CMBBE2004 Proceedings. FIRST Numerics Ltd.; 2005.

55. Sorkine O, Cohen-Or D, Lipman Y, Alexa M, Rössl C, Seidel HP. Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing. ACM; 2004. p. 175–184.

56. Drira H, Amor BB, Daoudi M, Srivastava A. Pose and expression-invariant 3d face recognition using elastic radial curves. In: British machine vision conference; 2010. p. 1–11.

57. Peyre G. Toolbox graph—A toolbox to process graph and triangulated meshes; Updated 27 Jun 2009. Available from: https://uk.mathworks.com/matlabcentral/fileexchange/5355-toolbox-graph/content/toolbox_graph/html/content.html.

58. Szeptycki P, Ardabilian M, Chen L. A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking. In: 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems. IEEE; 2009. p. 1–6.

59. Toma AM, Zhurov AI, Playle R, Marshall D, Rosin PR, Richmond S. The assessment of facial variation in 4747 British school children. European Journal of Orthodontics. 2012;34:665–664.

60. Besl PJ, McKay ND. Method for registration of 3-D shapes. In: Robotics-DL tentative. International Society for Optics and Photonics; 1992. p. 586–606.

61. Dudani SA. The distance-weighted k-nearest-neighbor rule. IEEE Transactions on Systems, Man, and Cybernetics. 1976;SMC-6(4):325–327.

62. Djordjevic J, Zhurov AI, Richmond S, Consortium V, et al. Genetic and environmental contributions to facial morphological variation: a 3D population-based twin study. PloS one. 2016;11(9):e0162250. doi: 10.1371/journal.pone.0162250 27584156

63. Gilani SZ, Mian A. Perceptual differences between men and women: A 3D facial morphometric perspective. In: Pattern Recognition (ICPR), 2014 22nd International Conference on. IEEE; 2014. p. 2413–2418.

64. Mpiperis I, Malassiotis S, Strintzis MG. 3-D Face Recognition With the Geodesic Polar Representation. IEEE Transactions on Information Forensics and Security. 2007;2(3-2):537–547.

65. Sethian JA. Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science. vol. 3. Cambridge university press; 1999.

66. Surazhsky V, Surazhsky T, Kirsanov D, Gortler SJ, Hoppe H; ACM. Fast exact and approximate geodesics on meshes. ACM transactions on graphics (TOG). 2005;24(3):553–560.

67. Xin SQ, Wang GJ. Improving Chen and Han’s algorithm on the discrete geodesic problem. ACM Transactions on Graphics (TOG). 2009;28(4):104.

68. Weber O, Devir YS, Bronstein AM, Bronstein MM, Kimmel R. Parallel algorithms for approximation of distance maps on parametric surfaces. ACM Transactions on Graphics (TOG). 2008;27(4):104.

69. Peyre G. Fast marching MATLAB toolbox; Updated 19 Jul 2009. Available from: https://uk.mathworks.com/matlabcentral/fileexchange/6110-toolbox-fast-marching.

70. Kirsanov D. Exact geodesic for triangular meshes; Updated 03 Mar 2008. Available from: https://uk.mathworks.com/matlabcentral/fileexchange/18168-exact-geodesic-for-triangular-meshes.

71. Bommes D, Kobbelt L. Accurate Computation of Geodesic Distance Fields for Polygonal Curves on Triangle Meshes. In: VMV. vol. 7; 2007. p. 151–160.

72. Meyer M, Desbrun M, Schröder P, Barr AH, et al. Discrete differential-geometry operators for triangulated 2-manifolds. Visualization and mathematics. 2002;3(2):52–58.

73. Goldfeather J, Interrante V. A novel cubic-order algorithm for approximating principal direction vectors. ACM Transactions on Graphics (TOG). 2004;23(1):45–63.

74. Fu JH, et al. Convergence of curvatures in secant approximations. Journal of Differential Geometry. 1993;37(1):177–190.

75. Alliez P, Cohen-Steiner D, Devillers O, Lévy B, Desbrun M; ACM. Anisotropic polygonal remeshing. ACM Transactions on Graphics (TOG). 2003;22(3):485–493.

76. Abbas H, Hicks Y, Marshall D, Zhurov AI, Richmond S. A 3D morphometric perspective for facial gender analysis and classification using geodesic path curvature features. Computational Visual Media. 2018; p. 1–16.

77. Howitt D, Cramer D. Introduction to statistics in psychology. Pearson education; 2007.

78. Law MT, Thome N, Cord M. Bag-of-words image representation: Key ideas and further insight. In: Fusion in Computer Vision. Springer; 2014. p. 29–52.

79. Ramyachitra D, Manikandan P. Imbalanced Dataset Classification and Solutions: A Review. International Journal of Computing and Business Research. 2014;5(4).

80. Burns N, Bi Y, Wang H, Anderson T. Sentiment analysis of customer reviews: Balanced versus unbalanced datasets. Knowledge-Based and Intelligent Information and Engineering Systems. 2011; p. 161–170.

81. Satyasree K, Murthy J. An exhaustive literature review on class imbalance problem. Int J Emerg Trends Technol Comput Sci. 2013;2:109–118.

82. Paris S. Multiclass GentleAdaboosting; Updated 24 Nov 2011. Available from: https://uk.mathworks.com/matlabcentral/fileexchange/22997-multiclass-gentleadaboosting.

83. Khan F. An initial seed selection algorithm for k-means clustering of georeferenced data to improve replicability of cluster assignments for mapping application. Applied Soft Computing. 2012;12(11):3698–3700.

84. Rendón E, Abundez I, Arizmendi A, Quiroz E. Internal versus external cluster validation indexes. International Journal of computers and communications. 2011;5(1):27–34.

85. Rendón E, Abundez IM, Gutierrez C, Zagal SD, Arizmendi A, Quiroz EM, et al. A comparison of internal and external cluster validation indexes. In: Proceedings of the 2011 American conference on applied mathematics and the 5th WSEAS international conference on computer engineering and applications, San Francisco, CA, USA. vol. 29; 2011. p. 158–163.

86. Liu Y, Li Z, Xiong H, Gao X, Wu J. Understanding of internal clustering validation measures. In: 2010 IEEE International Conference on Data Mining. IEEE; 2010. p. 911–916.

87. Stockburger DW. Multiple Regression with Categorical Variables;. Available from: http://www.psychstat.missouristate.edu/multibook/mlt08m.html.

88. Algina J, Olejnik S. Conducting power analyses for ANOVA and ANCOVA in between-subjects designs. Evaluation & the Health Professions. 2003;26(3):288–314.

89. Armstrong RA, Slade S, Eperjesi F. An introduction to analysis of variance (ANOVA) with special reference to data from clinical experiments in optometry. Ophthalmic and Physiological Optics. 2000;20(3):235–241.10897345

90. Chatfield C. Introduction to multivariate analysis. Routledge; 2018.

91. Serrano C, García-Fernández L, Fernández-Blázquez JP, Barbeck M, Ghanaati S, Unger R, et al. Nanostructured medical sutures with antibacterial properties. Biomaterials. 2015;52:291–300. doi: 10.1016/j.biomaterials.2015.02.039 25818435

92. Shrestha BK, Mousa HM, Tiwari AP, Ko SW, Park CH, Kim CS. Development of polyamide-6, 6/chitosan electrospun hybrid nanofibrous scaffolds for tissue engineering application. Carbohydrate polymers. 2016;148:107–114. doi: 10.1016/j.carbpol.2016.03.094 27185121

93. Tombari F, Salti S, Di Stefano L. Unique signatures of histograms for local surface description. In: European conference on computer vision. Springer; 2010. p. 356–369.

94. Guo Y, Bennamoun M, Sohel F, Lu M, Wan J, Kwok NM. A comprehensive performance evaluation of 3D local feature descriptors. International Journal of Computer Vision. 2016;116(1):66–89.

95. Kuhn M, Johnson K. Applied predictive modeling. vol. 26. Springer; 2013.

96. Burton AM, Bruce V, Dench N. What’s the difference between men and women? Evidence from facial measurement. Perception. 1993;22(2):153–176. doi: 10.1068/p220153

97. Seo H, Song Y, Kim C, Kim H. Characteristics of Korean Children’s Facial Anthropometry Evaluated by Three-dimensional Imaging. Journal of the International Society for Respiratory Protection Vol. 2016;33(1).

98. Gupta S, Markey MK, Bovik AC. Anthropometric 3D face recognition. International journal of computer vision. 2010;90(3):331–349.

99. Hamza AB, Krim H. Geodesic matching of triangulated surfaces. Image Processing, IEEE Transactions on. 2006;15(8):2249–2258.

100. Gatzke T, Grimm C. Improved curvature estimation on triangular meshes. Computer Science and Engineering, Washington University in St. Louis; 2004. WUCSE-2004–9. Available from: https://openscholarship.wustl.edu/cse_research/1056/.

101. Raschka S. About Feature Scaling and Normalization; Jul 11, 2014. Available from: https://sebastianraschka.com/Articles/2014_about_feature_scaling.html#about-standardization.

102. Anderson MJ, Legendre P. An empirical comparison of permutation methods for tests of partial regression coefficients in a linear model. Journal of statistical computation and simulation. 1999;62(3):271–303.

103. Bronstein AM, Bronstein MM, Kimmel R. Efficient computation of isometry-invariant distances between surfaces. SIAM Journal on Scientific Computing. 2006;28(5):1812–1836.

104. Vezzetti E, Marcolin F. 3D human face description: landmarks measures and geometrical features. Image and Vision Computing. 2012;30(10):698–712.

105. Gilani SZ, Rooney K, Shafait F, Walters M, Mian A. Geometric facial gender scoring: objectivity of perception. PloS one. 2014;9(6):e99483. doi: 10.1371/journal.pone.0099483 24923319

106. Brown I, Mues C. An experimental comparison of classification algorithms for imbalanced credit scoring data sets. Expert Systems with Applications. 2012;39(3):3446–3453.

107. Ng AY, Jordan MI, Weiss Y, et al. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems. 2002;2:849–856.

108. Galluccio L, Michel O, Comon P, Kliger M, Hero AO. Clustering with a new distance measure based on a dual-rooted tree. Information Sciences. 2013;251:96–113.

An automatic approach for classification and categorisation of lip morphological traits

Souhrn

Klíčová slova:

Zdroje

PLOS One

Aktuální možnosti diagnostiky a léčby litiáz