Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment

Paul Bakaki; Bruce Richard; Ella Pereira; Aristides Tagalakis; Andy Ness; Ardhendu Behera; Yonghuai Liu

doi:10.1007/978-3-031-06430-2_60

Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment

Paul Bakaki
, Bruce Richard
, Ella Pereira
, Aristides Tagalakis
, Andy Ness
, Ardhendu Behera
, Yonghuai Liu^*

^*Corresponding author for this work

Makerere University
Birmingham Women's and Children's NHS Foundation Trust
University of Bristol

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding (ISBN) › peer-review

153 Downloads (Pure)

Abstract

This paper proposes a novel method for the detection of the symmetrical axis of the cropped face required for the aesthetic outcome estimation from the facial images of patients after their cleft treatment. It firstly applies the Gaussian filter to smooth the images in order to compress noise on the subsequent tasks, then the bilateral semantic segmentation network is applied to segment the facial components out and each region is assigned a distinct colour, thirdly the Canny edge detector is applied to detect the facial feature points and all the contours are further detected and classified into three thirds according to their height. Fourthly, the centres of mass of detected feature points on the contours and the average of all these centres are used to estimate four potential symmetrical axes of the face, the one with minimum Manhattan distance from all the detected feature points is finally selected as the optimal one and used to estimate the aesthetic numerical score through the shape analysis in structural similarity measure. The experimental results based on a publicly accessible dataset shows that it performs well and better than one existing method.

Original language	English
Title of host publication	Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings
Editors	Stan Sclaroff, Cosimo Distante, Marco Leo, Giovanni M. Farinella, Federico Tombari
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	718-729
Number of pages	12
Volume	13232
ISBN (Print)	9783031064296
DOIs	https://doi.org/10.1007/978-3-031-06430-2_60
Publication status	Published - 17 May 2022
Event	21st International Conference on Image Analysis and Processing, ICIAP 2022 - Lecce, Italy Duration: 23 May 2022 → 27 May 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13232 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	21st International Conference on Image Analysis and Processing, ICIAP 2022
Country/Territory	Italy
City	Lecce
Period	23/05/22 → 27/05/22

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being
SDG 10 Reduced Inequalities

Keywords

Aesthetic outcome estimation
Cleft
Facial image
Shape analysis
Symmetrical axis

Access to Document

10.1007/978-3-031-06430-2_60

ICIAP2021Accepted author manuscript, 663 KBLicence: CC BY-NC-ND

Cite this

Bakaki, P., Richard, B., Pereira, E., Tagalakis, A., Ness, A., Behera, A., & Liu, Y. (2022). Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment. In S. Sclaroff, C. Distante, M. Leo, G. M. Farinella, & F. Tombari (Eds.), Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings (Vol. 13232, pp. 718-729). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13232 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-06430-2_60

Bakaki, Paul ; Richard, Bruce ; Pereira, Ella et al. / Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment. Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. editor / Stan Sclaroff ; Cosimo Distante ; Marco Leo ; Giovanni M. Farinella ; Federico Tombari. Vol. 13232 Springer Science and Business Media Deutschland GmbH, 2022. pp. 718-729 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{02bde6b7721c47248745ffff640825fc,

title = "Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment",

abstract = "This paper proposes a novel method for the detection of the symmetrical axis of the cropped face required for the aesthetic outcome estimation from the facial images of patients after their cleft treatment. It firstly applies the Gaussian filter to smooth the images in order to compress noise on the subsequent tasks, then the bilateral semantic segmentation network is applied to segment the facial components out and each region is assigned a distinct colour, thirdly the Canny edge detector is applied to detect the facial feature points and all the contours are further detected and classified into three thirds according to their height. Fourthly, the centres of mass of detected feature points on the contours and the average of all these centres are used to estimate four potential symmetrical axes of the face, the one with minimum Manhattan distance from all the detected feature points is finally selected as the optimal one and used to estimate the aesthetic numerical score through the shape analysis in structural similarity measure. The experimental results based on a publicly accessible dataset shows that it performs well and better than one existing method.",

keywords = "Aesthetic outcome estimation, Cleft, Facial image, Shape analysis, Symmetrical axis",

author = "Paul Bakaki and Bruce Richard and Ella Pereira and Aristides Tagalakis and Andy Ness and Ardhendu Behera and Yonghuai Liu",

note = "Funding Information: The facial images are the cropped and anonymised anteropos-terior (A/P) photos of 5-year-old children from the Cleft Care UK (CCUK). This publication presents data derived from the Cleft Care UK Resource (an independent study funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme RP-PG-0707-10034). PB was funded by Graduate Teaching Assistantship, Edge Hill University; YL was partially funded by Shaanxi Province Key Research and Development Plan General Project-Industrial Field (2021GY-171). Funding Information: Acknowledgments. The facial images are the cropped and anonymised anteroposterior (A/P) photos of 5-year-old children from the Cleft Care UK (CCUK). This publication presents data derived from the Cleft Care UK Resource (an independent study funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme RP-PG-0707-10034). PB was funded by Graduate Teaching Assistantship, Edge Hill University; YL was partially funded by Shaanxi Province Key Research and Development Plan General Project-Industrial Field (2021GY-171). Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 21st International Conference on Image Analysis and Processing, ICIAP 2022 ; Conference date: 23-05-2022 Through 27-05-2022",

year = "2022",

month = may,

day = "17",

doi = "10.1007/978-3-031-06430-2\_60",

language = "English",

isbn = "9783031064296",

volume = "13232",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "718--729",

editor = "Stan Sclaroff and Cosimo Distante and Marco Leo and Farinella, \{Giovanni M.\} and Federico Tombari",

booktitle = "Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings",

address = "Germany",

}

Bakaki, P, Richard, B, Pereira, E , Tagalakis, A, Ness, A, Behera, A & Liu, Y 2022, Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment. in S Sclaroff, C Distante, M Leo, GM Farinella & F Tombari (eds), Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. vol. 13232, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13232 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 718-729, 21st International Conference on Image Analysis and Processing, ICIAP 2022, Lecce, Italy, 23/05/22. https://doi.org/10.1007/978-3-031-06430-2_60

Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment. / Bakaki, Paul; Richard, Bruce; Pereira, Ella et al.
Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. ed. / Stan Sclaroff; Cosimo Distante; Marco Leo; Giovanni M. Farinella; Federico Tombari. Vol. 13232 Springer Science and Business Media Deutschland GmbH, 2022. p. 718-729 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13232 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding (ISBN) › peer-review

TY - GEN

T1 - Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment

AU - Bakaki, Paul

AU - Richard, Bruce

AU - Pereira, Ella

AU - Tagalakis, Aristides

AU - Ness, Andy

AU - Behera, Ardhendu

AU - Liu, Yonghuai

N1 - Funding Information: The facial images are the cropped and anonymised anteropos-terior (A/P) photos of 5-year-old children from the Cleft Care UK (CCUK). This publication presents data derived from the Cleft Care UK Resource (an independent study funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme RP-PG-0707-10034). PB was funded by Graduate Teaching Assistantship, Edge Hill University; YL was partially funded by Shaanxi Province Key Research and Development Plan General Project-Industrial Field (2021GY-171). Funding Information: Acknowledgments. The facial images are the cropped and anonymised anteroposterior (A/P) photos of 5-year-old children from the Cleft Care UK (CCUK). This publication presents data derived from the Cleft Care UK Resource (an independent study funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme RP-PG-0707-10034). PB was funded by Graduate Teaching Assistantship, Edge Hill University; YL was partially funded by Shaanxi Province Key Research and Development Plan General Project-Industrial Field (2021GY-171). Publisher Copyright: © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2022/5/17

Y1 - 2022/5/17

N2 - This paper proposes a novel method for the detection of the symmetrical axis of the cropped face required for the aesthetic outcome estimation from the facial images of patients after their cleft treatment. It firstly applies the Gaussian filter to smooth the images in order to compress noise on the subsequent tasks, then the bilateral semantic segmentation network is applied to segment the facial components out and each region is assigned a distinct colour, thirdly the Canny edge detector is applied to detect the facial feature points and all the contours are further detected and classified into three thirds according to their height. Fourthly, the centres of mass of detected feature points on the contours and the average of all these centres are used to estimate four potential symmetrical axes of the face, the one with minimum Manhattan distance from all the detected feature points is finally selected as the optimal one and used to estimate the aesthetic numerical score through the shape analysis in structural similarity measure. The experimental results based on a publicly accessible dataset shows that it performs well and better than one existing method.

AB - This paper proposes a novel method for the detection of the symmetrical axis of the cropped face required for the aesthetic outcome estimation from the facial images of patients after their cleft treatment. It firstly applies the Gaussian filter to smooth the images in order to compress noise on the subsequent tasks, then the bilateral semantic segmentation network is applied to segment the facial components out and each region is assigned a distinct colour, thirdly the Canny edge detector is applied to detect the facial feature points and all the contours are further detected and classified into three thirds according to their height. Fourthly, the centres of mass of detected feature points on the contours and the average of all these centres are used to estimate four potential symmetrical axes of the face, the one with minimum Manhattan distance from all the detected feature points is finally selected as the optimal one and used to estimate the aesthetic numerical score through the shape analysis in structural similarity measure. The experimental results based on a publicly accessible dataset shows that it performs well and better than one existing method.

KW - Aesthetic outcome estimation

KW - Cleft

KW - Facial image

KW - Shape analysis

KW - Symmetrical axis

UR - https://www.scopus.com/pages/publications/85130880714

UR - https://www.scopus.com/pages/publications/85130880714#tab=citedBy

UR - https://www.mendeley.com/catalogue/300225a4-1a87-3104-b893-75630b828a6a/

U2 - 10.1007/978-3-031-06430-2_60

DO - 10.1007/978-3-031-06430-2_60

M3 - Conference proceeding (ISBN)

AN - SCOPUS:85130880714

SN - 9783031064296

VL - 13232

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 718

EP - 729

BT - Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings

A2 - Sclaroff, Stan

A2 - Distante, Cosimo

A2 - Leo, Marco

A2 - Farinella, Giovanni M.

A2 - Tombari, Federico

PB - Springer Science and Business Media Deutschland GmbH

T2 - 21st International Conference on Image Analysis and Processing, ICIAP 2022

Y2 - 23 May 2022 through 27 May 2022

ER -

Bakaki P, Richard B, Pereira E , Tagalakis A, Ness A, Behera A et al. Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment. In Sclaroff S, Distante C, Leo M, Farinella GM, Tombari F, editors, Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. Vol. 13232. Springer Science and Business Media Deutschland GmbH. 2022. p. 718-729. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2022 May 17. doi: 10.1007/978-3-031-06430-2_60

Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Translating Automated Facial Appearance Assessment Model into Clinical Practice Tool for Cleft Lip Outcome Evaluation

Cite this

Key Landmarks Detection of Cleft Lip-Repaired Partially Occluded Facial Images for Aesthetics Outcome Assessment

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Impacts

Translating Automated Facial Appearance Assessment Model into Clinical Practice Tool for Cleft Lip Outcome Evaluation

Cite this