Computational Approach for an Automatic Facial Appearance Outcome Measure of Cleft Lip Surgical Repair Using Digital Images

Abstract

Cleft lip (CL) is a common congenital facial anomaly that affects several individuals worldwide and is treated through a surgical procedure. The appearance outcome following the procedure is normally qualitatively assessed by human experts. However, human experts are naturally constrained by fatigue, potential bias, replicability weaknesses, and inconsistencies. The presence of large datasets presents further challenges to human qualitative assessment.
This study aims to develop and validate novel computational techniques that can automatically, objectively, and quantitatively assess the CL treatment outcome. Consequently, the study assesses the effectiveness of CL treatment using computational techniques. Using digital imagery, this study has led to the development and validation of some computational techniques to aid with automatic, objective, and quantitative assessment CL treatment outcome.

The first approach investigated the appearance and shape of the mouth lips as a region of interest for analysis. The bisector of the line connecting the mouth corners was estimated as the vertical symmetric axis of the mouth borderline. By splitting the mouth blob into two parts, the two parts were analysed for structural similarity. Consequently, a numeric score ranging from 1 to 5 was generated and validated using Pearson correlation coefficient against human-assigned numeric scores.
Secondly, a novel technique for adaptive detection of the symmetric axis of the cropped facial images of patients after CL treatment was developed. A Gaussian filter was applied to smoothen the images to compress potential noise on the subsequent tasks. Segmentation using a bilateral semantic network was applied to detect the facial components in each region of interest in the facial image. Applying the previous approach led to improved validation metrics using Pearson’s correlation coefficient.
The final approach explored transfer learning using CNNs in a regression analysis study. An investigation was completed for the impact of transfer learning on regression scoring and assessed its potential in overcoming dataset limitation challenges. Through extensive experimentation and evaluation of diverse regression scoring combinations, different numeric assessment prediction results were generated. It was demonstrated that appearance assessment through CNN transfer learning is significantly competitive and better than human expert assessment and scoring. Competitive metrics using RMSE, MAE, and Pearson correlation were generated.

Overall, this thesis presents a comprehensive computational approach for automatic appearance assessment estimation of CL treatment using digital imagery. It offers insights into the potential of advanced computational techniques, such as shape analysis and deep learning, to provide accurate and objective assessments. The findings contribute to the field of CL treatment evaluation and pave the way for further advancements in automated appearance assessment methodologies.

Date of Award	1 May 2024
Original language	English
Awarding Institution	Edge Hill University
Supervisor	YONGHUAI LIU (Director of Studies), ELLA PEREIRA (Supervisor), ARISTIDES TAGALAKIS (Supervisor) & Bruce Richard (Supervisor)