 |
|
Measurement of Facial Movement With Computer Software
Eric W. Sargent, MD;
Omar A. Fadhli, MD;
Randall S. Cohen, MD
Arch Otolaryngol Head Neck Surg. 1998;124:313-318.
ABSTRACT
| |
Objective To adapt desktop computer software to objectively grade facial movement.
Design The criteria of the facial nerve grading system by House and Brackmann, the current "gold standard," are prone to ambiguous interpretation. Proposed objective grading systems compare the movement of points on each side of the face or use subtraction and thresholding of digitized images of the face to yield images that represent moving areas of the face. The movement of a point on the face and the area of motion determined by digital subtraction were compared during an increasing smile in healthy subjects. The Nottingham system (calculated using measurement of the movement of 4 points on the face) using desktop computer software (Adobe Photoshop 3.0, Adobe Systems Inc, Mountain View, Calif) to measure movement of the points was compared with the system by House and Brackmann. The computer software was used to subtract digitized images and derive a facial movement score, which was compared with the scores of the systems by Nottingham and House and Brackmann.
Setting Academic otologic practice.
Study Participants Nine patients with varying degrees of facial nerve disability and 7 individuals with normal facial nerve function.
Results The movement of the oral commissure compared with the apparent area of movement of the face determined by digital subtraction had high intersubject variability. In patients with facial weakness, the Nottingham score had a correlation coefficient of -0.97 compared with the House and Brackmann grade, and the digital subtraction score had a correlation coefficient of -0.62 (paired Student t test).
Conclusions The desktop computer software can be used to calculate the Nottingham score. Digital subtraction as a measure of facial function warrants further study.
INTRODUCTION
IN 1985, Pearson1 wrote of facial nerve grading: "Subjectivity cannot be eliminated, and so complex a function cannot be readily simplified." To a great degree his words have proved true, although many have worked to make the process of facial nerve grading objective.
Ideally, a facial nerve grading system should be continuous (rather than discontinuous, where the results are divided into discrete categories), easy to use, and unambiguous. Using such a system, reports of outcomes of surgery (eg, facial nerve injury following acoustic neuroma surgery) or interventions (eg, steroid use in Bell palsy) would be more easily evaluated and observer bias would be reduced. House and Brackmann's2 (H-B) system, the most common and easy to use system, is discontinuous, subjective, and ambiguous.
Other systems have been proposed to replace the H-B system. Burres and Fisch3-4 developed a system based on a study of facial biomechanics. While it is objective and continuous, this system is time-consuming and cumbersome.5-6 Critics6 of the system have also noted that it tends to underestimate the degree of dysfunction in severe paralysis and overestimate it in mild paralysis.
In 1994, Murty et al6 described the Nottingham system of measuring facial nerve function. It is calculated by measuring the movement of 4 points on the face and comparing the abnormal side with the normal side as a percentage. It gives a global percentage scale that is continuous and has been validated by the Murty et al against the H-B system. In the Nottingham system, issues of secondary defects are addressed in parts 2 and 3 of a 3-part score in a manner similar to the TNM scale used in oncology. Figure 1 shows the reference points measured in the Nottingham system, and Figure 2 shows how the Nottingham score is derived.
|
|
|
|
Figure 1. Facial reference points for the Nottingham system.6 SO indicates superior orbital rim; LC, lateral canthus; IO, inferior orbital rim; and M, oral commissure.
|
|
|
|
|
|
|
Figure 2. The Nottingham facial nerve grading system.6
|
|
|
In 1992, Neely and associates7 described a computerized image analysis method. By subtracting digitized video images of the face in motion from an image of the face at rest, they derived an image that makes areas of the face that change their gray scale value appear bright. By thresholding (converting all pixels above a certain value to white and those below the value to black), areas that appear to approximate the parts of the face in motion are highlighted in white and still areas are black. With analysis of sequential video images, Neely et al derived intensity-duration curves. They found that a score derived using this analysis correlated with the H-B score. To date, the software used in their study has been unavailable to the public.
In 1996, Neely and associates8 updated their grading system by adding a subjective observation of specific areas of the face (brow, eye, and mouth).These observations are limited to 4 categories (normal, less than normal without facial dyskinesia, less than normal with facial dyskinesia, and no movement). The subjective component is then added to a 2-part computer-derived score for each area of the face to arrive at a composite index. Using the pixel subtraction method, the first part of the computer-derived score assigns the area to 1 of 5 types of intensity-duration plots. The second computer-derived score is based on the maximum amplitude of the computed curve. The scores are added by region and summed to arrive at a composite index for each half of the face. Interestingly, the subjective component is given the most weight (>80%) in the composite index, negating the benefits of a computerized objective grading scale and making this a primarily subjective grading scale.
We sought to develop a computer-based method of objective analysis. Our goals were to use widely available, reasonably inexpensive software for a desktop system to objectively grade facial movement. We also sought a method that would be facilitated by but not require the use of a computer. We also wished to explore the possibility of pixel subtraction to analyze facial movement.
The computer software (Adobe Photoshop 3.0, Adobe Systems Inc, Mountain View, Calif) is the industry standard for computerized image manipulation, chiefly used by graphic artists. Among its many features, Photoshop allows image subtraction and the counting of pixels in areas of images. After determining that pixel subtraction could be performed using Photoshop, we studied the relationship between a 2-dimensional movement of a point on the face and the number of pixels derived by the pixel subtraction method in that area of the face in control subjects. We then studied a group of patients with facial nerve problems. We compared point-to-point movement analysis to grade facial nerve function with the pixel subtraction method.
SUBJECTS, MATERIALS, AND METHODS
SUBJECTS
Seven subjects with no history of facial nerve problems were studied as a control group. Nine subjects with varying degrees of facial nerve function from a variety of causes were studied. Written permission for the use of facial photographs in publications was obtained from all individuals.
MATERIALS
A still camera (Canon EOS 650, Canon, Lake Success, NY)fitted with a ring flash was used to photograph the subjects. The camera was fixed to a slitlamp examination table that stabilized the head to keep a constant relationship between the camera and subject. Images were transferred to CD-ROM (Kodak PhotoCD, Eastman Kodak Co, Rochester, NY). Photoshop running on a computer (Macintosh Centris 650, Apple, Cupertino, Calif) was used to perform image analysis.
METHODS
Control Subjects
The 7 control subjects used to study the relationship between the distance moved by a point on the face and the area represented by the pixel subtraction method were photographed in a darkened room. Faces were illuminated only by the ring flash. Adhesive marks were positioned at the oral commissure, the lateral canthus, the inferior orbital rim, and the superior orbital rim (Figure 1). A sequence of photographs was taken of a progressively increasing smile, starting at rest.
After transfer to CD-ROM, the image of the face at rest (Figure 3, A) was viewed using Photoshop at a resolution of 512x768. The x and y coordinates of the marker at the oral commissure were then read using the Info Palette. An image of the smiling face (Figure 3, B) was then placed on top of the face at rest. Using the Layers palette, the overlaid image was made semiopaque (Figure 3, C) and placed so that constant reference points (in this case, the corneal light reflex as well as the rulers on the slitlamp examination table) were aligned (Figure 3, D). The overlaid image was then made opaque and the x and y coordinates of the marker at the oral commissure were read using the Info Palette. Using a spreadsheet (Microsoft Excel, Microsoft Corporation, Redmond, Wash), the movement of the marker was calculated using the Pythagorean theorem. Figure 3, E, shows how position of the markers can be calculated precisely using Photoshop.
|
|
|
|
Figure 3. A, Rest; B, full smile; C, overlaid semiopaque images; D, overlaid semiopaque images nudged into position; and E, determination of marker position.
|
|
|
Pixel subtraction was performed using Photoshop by first converting the color images to gray scale with the Image-Adjust-Desaturate command. The image of the smiling face was then laid on top of the face at rest and aligned as described previously. After alignment, the images were subtracted using the Difference command in the Layers palette. After clearing the Marquee tool, the result was subjected to thresholding using the Image-Map-Threshold command. A threshold of 30 was selected. Like Neely et al,7 we found little difference in the result when different threshold levels were used. Figure 4 shows the result of these image manipulations for the images used in Figure 3. The number of pixels in a given area can be determined by outlining the area of interest with the Marquee or Lasso tool and then counting the white pixels using the Image-Histogram command. This last step was performed 3 times, and the average was used in analysis.
|
|
|
|
Figure 4. Pixel subtraction of Figure 3, D.
|
|
|
Subjects With Impaired Facial Nerve Function
Subjects with impaired facial nerve function were photographed in a manner identical to that used for the control subjects. Three standard facial movements were studied: full smile, eye closure, and forehead wrinkling. The position of the facial markers was calculated by the method described and the Nottingham score calculated using the spreadsheet. On the identical images, the pixel subtraction method was used. The number of pixels in 3 zones (Figure 5) was determined for each half of the face. The sum of the pixels in zones 1, 2, and 3 of the impaired side was divided by that on the normal side and the result expressed as a percentage of relative movement.
|
|
|
|
Figure 5. Facial zones (1, 2, and 3) used in pixel subtraction measurement of facial movement.
|
|
|
RESULTS
CONTROL SUBJECTS
Figure 6 shows how 2-dimensional movement of a marker of the face corresponds to the area of movement derived by the pixel subtraction method.
|
|
|
|
Figure 6. Movement of oral commissure marker vs pixels in zone 1 determined by pixel subtraction during a progressive smile in control subjects. Different symbols represent different individuals. Paired symbols and lines represent each half of the face.
|
|
|
We found that while progressive movement of the marker at the corner of the mouth was associated with an increase in the pixel area, the variation between individuals made generalization of a relationship between the 2 variables impossible. We found that there was little overall side-to-side variability. We observed that subjects with facial hair or coarse skin were among those with the steepest slopes, suggesting that highly textured surfaces may increase the apparent area of movement.
To determine if movement in the anterior-posterior direction (z axis) not measured by our 2-dimensional photographic technique may explain the variability between the movement of the marker and the result of pixel subtraction, the movement of the oral commissure on the x, y, and z axes was manually measured for 6 control subjects. The relationship between the movement on the x and y axes and the z axis was linear with a regression coefficient of 0.92 (R2=0.85).
SUBJECTS WITH IMPAIRED FACIAL NERVE FUNCTION
Figure 7 shows the performance of the pixel subtraction method compared with the H-B scale in subjects with impaired facial nerve function. The correlation coefficient is -0.62 (R2=0.38).
|
|
|
|
Figure 7. House and Brackmann2 (H-B) scores vs summed pixel subtraction results during full smile, eye closure, and brow raising in subjects with impaired facial nerve function. The circles indicate individual subjects; line, linear regression.
|
|
|
Figure 8 shows the performance of the Nottingham system with the score calculated using coordinates determined by Photoshop compared with the H-B scale. The correlation coefficient is -0.97 (R2=0.94).
|
|
|
|
Figure 8. House and Brackmann2 (H-B) scores vs computer-derived Nottingham6 scores in subjects with impaired facial nerve function. The circles indicate individual subjects; line, linear regression.
|
|
|
COMMENT
Pixel subtraction methods for automation of facial nerve grading is an excellent goal; however, the results have not been ideal. In our study of control subjects, we found high intersubject variability when we compared the linear movement of a point on the face with the area of movement depicted by pixel subtraction. The results suggest that the relationship between the movement of a point on the face and the number of pixels derived using pixel subtraction and thresholding is complicated and highly individual.
Comparing the pixel subtraction method of estimating facial movement with the movement of a single point in the center of the area studied is complicated by several factors. Movement of the point measured with the computer is only in the plane of the x and y axes (measurement of movement in the z axis requires 2 additional cameras, one for each half of the face) and is 2-dimensional, giving rise to what may be an unfair comparison to the 3-dimensional pixel subtraction method. However, the linear relationship we found between the movement of the oral commissure on the x and y axes and the z axis suggests that the z axis component may not be responsible for the marked intersubject variability we found when comparing the pixel area with the movement on the x- and y-axes. Comparing the movement of a single point with the area of change represented by the pixel subtraction method may not be valid because the conformation of the face surrounding point M (Figure 1) may change even after the point M has moved its farthest.
The reliance of pixel subtraction on movement of the entire draped skin surface rather than linear movement of standardized points on the face may cause error in measurement. For example, in the case of extreme laxity of the skin and superficial musculoaponeurotic system we have observed that movement of the half of the face depicted by pixel subtraction may underestimate facial movement. Interestingly, this underestimation may not hold true for grading systems based on linear point motion. Since the measured points (inferior and superior orbit and oral commissure) are near areas of muscular insertion, significant measurable movement of these points still occurs.
Overestimation of facial nerve function by pixel subtraction occurs in 2 situations. First, minute displacement of irregular facial features may falsely amplify actual movement. We observed this in individuals with facial hair and coarsely textured skin. Second, because the normal side of the face pulls the half of the face that is completely paralyzed, actual movement may be overestimated. In many patients with severe paralysis, this point proved to be the most confounding factor since pixel subtraction does not distinguish disorganized movement from organized, directional movement. Figure 9, A through C, which shows an elderly subject, illustrates this point. There is marked pulling of the paralyzed side by the normal side.
|
|
|
|
Figure 9. Elderly subject with complete facial palsy demonstrating errors in the pixel subtraction system due to skin laxity and facial hypomobility. Little movement appears on the normal side (subject's right) of the face and there is movement of the patient's atonic side (left) due to the pull from the normal side. A, Rest; B, full smile; C, pixel subtraction of A and B.
|
|
|
Apparent voluntary eye closure is often observed in patients with complete facial paralysis due to tissue elasticity and the pull of gravity on the upper lid. We observed that this phenomenon confounded the pixel subtraction method we used by artificially augmenting the scores of patients with complete facial paralysis. A successful pixel-based system will need to reject this area of the face to avoid this error.
Computer-aided determination of the Nottingham grade offers several advantages over the manual measurement and computation of the grade. Using the computer, the Nottingham score can be calculated at leisure from a permanent image of the patient and measurements can be taken from more than one area on the face at the same time, which is difficult to do with a ruler or calipers.
In this article, we have not compared the accuracy of computer-aided determination with manual determination of the Nottingham grade. Our experience with the 9 subjects with impaired facial nerve function presented herein suggests that the computerized determination of the Nottingham score correlates well with the manually measured score. More data on patients with impaired facial nerve function are being accrued to allow an accurate comparison in a future report.
CONCLUSION
Photoshop can be used to perform the pixel subtraction analysis described by Neely et al,7 as well as the analyses we describe. Although Photoshop approximates the system used by Neely et al, it does not duplicate that system. Nonetheless, currently there are inherent faults with the pixel subtraction method that are eliminated with other grading systems. We believe that the Nottingham system provides a relatively simple, continuous scale. While the computerization we used has the advantage of using commercial software and hardware that are relatively inexpensive, the Nottingham system can be performed without a computer, allowing any practitioner to use it to report results.
AUTHOR INFORMATION
Accepted for publication September 12, 1997.
Presented at the Second International Congress on Skull Base Surgery, San Diego, Calif, July 3, 1996.
Reprints: Eric W. Sargent, MD, Department of Otolaryngology–Head and Neck Surgery, St Louis University School of Medicine, 3635 Vista Ave, Box 15250, St Louis, MO 63110-0250 (e-mail: sargente{at}slu.edu).
From the Department of Otolaryngology–Head and Neck Surgery, St Louis University School of Medicine, St Louis, Mo. We have received no support from Adobe Systems Inc for the work presented herein.
REFERENCES
| |
1. Pearson BW. Editorial. Otolaryngol Head Neck Surg. 1985,93:145.
2. House JW, Brackmann DE. Facial nerve grading system. Otolaryngol Head Neck Surg. 1985;93:146-147.
WEB OF SCIENCE
| PUBMED
3. Burres S. Facial biomechanics: the standard of normal. Laryngoscope. 1985;95:708-714.
WEB OF SCIENCE
| PUBMED
4. Burres S, Fisch U. The comparison of facial grading systems. Arch Otolaryngol Head Neck Surg. 1986;112:753-758.
FREE FULL TEXT
5. Croxson G, May M, Mester SJ. Grading facial nerve function: House-Brackmann vs Burres-Fisch. Am J Otol. 1990;11:240-246.
WEB OF SCIENCE
| PUBMED
6. Murty GE, Diver JP, Kelly PJ, O'Donoghue GM, Bradley PJ. The Nottingham system: objective assessment of facial nerve function in the clinic. Otolaryngol Head Neck Surg. 1994;110:156-161.
WEB OF SCIENCE
| PUBMED
7. Neely JG, Cheung JY, Wood M, Byers J, Rogerson A. Computerized quantitative dynamic analysis of facial motion in the paralyzed and synkinetic face. Am J Otol. 1992;13:97-107.
WEB OF SCIENCE
| PUBMED
8. Neely JG, Joaquin AH, Kohn LA, Cheung JY. Quantitative assessment of the variation within grades of facial paralysis. Laryngoscope. 1996;106:438-442.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Evaluation of periapical changes following endodontic therapy: digital subtraction technique compared with computerized morphometric analysis
Carvalho et al.
Dentomaxillofac Radiol 2009;38:438-444.
ABSTRACT
| FULL TEXT
Outcome Measures in Facial Plastic Surgery: Patient-Reported and Clinical Efficacy Measures
Rhee and McMullin
Arch Facial Plast Surg 2008;10:194-207.
ABSTRACT
| FULL TEXT
Agreement between clinical and electromyographic assessments during the course of peripheric facial paralysis
On et al.
Clin Rehabil 2007;21:344-350.
ABSTRACT
Videomimicography: The Standards of Normal Revised
Dulguerov et al.
Arch Otolaryngol Head Neck Surg 2003;129:960-965.
ABSTRACT
| FULL TEXT
|
