Since 2004, the Geena Davis Institute on Gender in Media at Mount Saint Mary’s University has advocated for greater inclusion in entertainment media through cutting-edge research and advocacy. The Institute is moving the needle on intersectional gender representation by working directly within the industry, with a particular focus on children’s entertainment. This report analyzes representations of gender, race, LGBTQ+, disability, age (over 60) and body size (large) in the top-grossing family films (rated G, PG, or PG-13) of 2019. We also include analysis of leading characters in live action and animated family films from over a decade (2007-2019). Here are our key findings:

GENDER

• We have achieved another historic milestone: for the first time, lead female characters have reached parity in the top-100 grossing family films! The percentage of female leads in family films has doubled from 2007 (24%) to 2019 (48%).
• We see signs of progress with female characters’ speaking time in family films. The percentage of speaking time has increased from 31.3% in 2014 to just under 40% in 2019.
• The percentage of female characters’ screen time has also increased nearly 10 points since 2014—from 34.9% to 42.6% in 2019.
• When it comes to female leading characters, 67.3% are white women and 32.6% are women of color. People of color are 38% of the U.S. population, which means that women of color are still underrepresented in leading roles.
• We have not seen much progress with gender and supporting characters. Male supporting characters still outnumber female supporting characters 2-to-1.
• Female characters are six times more likely than male characters to be shown in revealing clothing (18.6% compared with 2.7%).
• With age, a majority of female characters in family films are under age 40 (60.2%), while a majority of male characters are age 40+ (56.4%) in 2019.

RACE/ETHNICITY

• We see signs of progress for people of color in leading roles. The percentage of leads of color has increased dramatically in the past decade to a historic high of 30% in 2019. This is up from 22% in 2007.
• White characters are more likely than characters of color to be portrayed as having higher socioeconomic status (“upper-class”) in family films (14.8% compared with 8.7%).
• White characters and characters of color are equally likely to be shown as having an occupation. No significant differences are found in terms of professional, managerial, STEM, or other occupation types.
• White characters are more likely to be shown as leaders than characters of color (46.7% compared with 39.8%).

LGBTQ+

• Only 2% of family films in 2019 featured a leading LGBTQ+ character, down from a historic high of 5% in 2018.
• LGBTQ+ characters are more likely than heterosexual characters to be shown in revealing clothing (27.8% compared with 8.8%).
• LGBTQ+ characters are more likely to die in family films than heterosexual characters (16.7% compared to 11.9%).
• A greater percentage of LGBTQ+ characters are portrayed as sexually promiscuous than heterosexual characters (5.6% compared with 1.6%).
• Heterosexual characters are more likely to be portrayed as hard working than LGBTQ+ characters (51.9% compared with 33.3%).
• Heterosexual characters are more likely to be portrayed as smart than LGBTQ+ characters (41.6% compared to 27.8%).

DISABILITY

• We see signs of progress with disability representations. Eight percent of family films featured a lead with a disability in 2019—a historic high.
• Characters with disabilities are more likely to be rescued in family films than other characters (34.3% compared with 20.6%).
• Characters with disabilities are nearly twice as likely to die in family films as other characters (20.0% compared with 11.7%).
• Characters with disabilities are more likely to be represented as hard working, in management positions, in STEM occupations, and as leaders than other characters.

AGE (60+)

• People 60+ make up 19% of the American public, but only 9.1% of characters in family films and only 3.5% of leading characters.
• Characters age 60+ are more likely to be shown with zero sexual partners than younger characters (97.3% compared with 87.2%). This gap reinforces the stereotype that elderly people are non-sexual.
• Only 39.1% of characters ages 60+ are shown as having an occupation compared with 80.7% of younger characters.
• Characters 60+ are more likely to be shown in management positions and as leaders in the workplace than younger characters.

BODY SIZE (LARGE BODY TYPE)

• Overall, only 8.3% of characters are large, which means this identity group is underrepresented in family films as compared with the broader population.
• A sizeable number of large characters are presented as damaging stereotypes—lazy (7.4%), physically slow (8.5%), stupid (9.6%), poorly dressed (8.5%), clumsy (7.4%), and as a punchline (9.6%) in family films.
• Large characters are less likely to be shown as smart than other characters (35.1% compared with 42.1%).
• When it comes to verbal shaming from other characters or narrators, 7.4% of large characters are verbally shamed in family films.

GENDER

RACE/ETHNICITY

LGBTQ+

DISABILITY

AGE

BODY SIZE

This study is critical because the stories that we choose to tell in entertainment media send a specific message about who matters most in our culture. In order to bring about a global culture change, it is especially important that children see diverse, intersectional representations of characters in media to reflect the population of the world—which is half female and very diverse— and avoid unwittingly instilling unconscious bias in them. As the historical findings of this report indicate, content creators in family film are making significant progress when it comes to inclusion.

Our research focuses on children’s and family programming in order to assess how media is impacting young people because youth are the highest consumers of media and the group most impacted by media content. A report from Common Sense Media finds that tweens use an average of six hours of entertainment media per day, while teens use an average of 9 hours per day.¹ Children are particularly impacted by not seeing characters like themselves reflected in popular culture, as they are in the process of developing their identity and finding their place in the world.

Our research is unique in its approach and methods. First, we focus on family content. Second, our team of expert human coders applies a double-coding technique that increases the validity and reliability of the findings. Third, we are the only public data institute to consistently analyze representations of the six major marginalized identities: women, people of color, LGBTQ+ individuals, people with disabilities, older people (60+), and large-bodied individuals.

Additionally, our research organization uses the Geena Davis Inclusion Quotient (GD-IQ), a software tool with the ability to measure screen and speaking time through the use of automation (see Appendix A). This revolutionary machine learning tool was developed by the Institute and funded by Google.org. The GD-IQ, which incorporates machine learning technology, was designed by Dr. Shrikanth Narayanan and his team of researchers at the University of Southern California’s Signal Analysis and Interpretation Laboratory (SAIL), along with Dr. Caroline Heldman, Vice President for Research and Insights for the Institute.

FINDINGS

In this section, we summarize our major findings for character representation by gender, race/ethnicity, sexuality (LGBTQ), disability, age (60+), and body size (large).

GENDER

In this section, we examine gender and representation in terms of character prominence, age, stereotypes, work and leadership, traits, and box office returns.

Prominence

• We have achieved another historic milestone: for the first time, lead female characters have reached parity in the top-100 grossing family films! The percentage of female leads has increased significantly in family films in the past decade— from 24% in 2007 to 48% in 2019.
• The percentage of female leads in the top family films doubled from 2007 (24%) to 2019 (48%).
• For 2019, male characters outnumber female characters two-to-one (67.2% compared to 32.0%). This gap is found with supporting characters (38.4% female, 61.2% male) and minor characters (33.1% female, 66.5% male).² This means that even though we have achieved gender parity for leads, further progress is needed to make sure the fictitious worlds created reflect the population.

GD-IQ: Screen Time and Speaking Time

• We see signs of progress with female characters’ speaking time in family films. The percentage of speaking time has increased from 31.3% in 2014 (when we first started to measure it) to just under 40% for the past two years. This is a significant increase in the on-screen presence of female characters, as measured by the percentage of time they are speaking compared with male characters.
• The percentage of female characters’ screen time has also increased nearly 10 points since 2014—from 34.9% to 42.6%.

Stereotypes

Objectification and sexualization are remarkably common in family film content:

• Female characters are six times more likely than men to be shown in revealing clothing (18.6% compared with 2.7%).
• Female characters are three times as likely to be shown partially nude as male characters (9.8% compared with 3.2%).
• Visual objectification occurred among 8.0% of female characters compared with 1.9% of male characters, and verbal objectification was three times more common for female characters than male characters (6.0% compared with 1.9%).³

We also examined characteristics that are typically associated with masculinity—factors such as aggression and risk taking:⁴

• In family film, male characters are more likely to be shown as violent (35.2%) than female characters (24.6%).
• Male characters are also more than twice as likely as female characters to be shown as criminal (18.6% compared with 7.8%).
• Male characters are more likely to die in family films than female characters (13.0% compared with 9.8%).

Work and Leadership

Representations of work and leadership in family films tend to reinforce stereotypes about gendered occupations and men as breadwinners:

• In family film, female characters are more likely than male characters to be shown without an occupation (26.4% compared with 16.9%).
• Male characters are more likely to be shown working in the military (9.0% compared with 3.3%) and in criminal occupations (e.g., trafficker, bank robber; 5.4% compared with 2.0%)
• Female and male characters are equally likely to be depicted in STEM professions.
• Male characters are more likely to be shown in positions of leadership than female characters (46.4% compared with 41.5%).

Character Traits

We do not find gender differences in characters traits that fit with gendered stereotypes:

• In family films, female characters are more likely to be shown as smart than male characters (45.5% compared with 38.4%).
• There are no gender differences in representations of humor in the top family films of 2019.

Box Office

Women make up 51% of film-going audiences.⁵ A recent study debunks the idea that movies with a female lead or a lead of color earn less money at the box office, and find that box office returns are instead driven by distribution, marketing, the quality of the story, and marketing and production costs.⁶ We find few gender differences in box office returns for family films.

• Box office proceeds from family films with female leads have seen a general increase over time, peaking in 2016 at an average of $94.3 million.
• Family films with female leads have typically generated more domestic box office revenue than family films led by men in recent years, but in 2019, male-led family films grossed significantly more on average than female-led family films ($110.6 million compared to $64.1 million).
• When it comes to worldwide box office figures, family films with female leads have steadily increased in revenue, hitting a historic high averaging $279.6 million in 2019.
• Family films with female leads are closing the worldwide gender gap with male leads, but family films with male leads still earn significantly more on average in box office revenues.

Intersectional Analysis

• When it comes to female leading characters, 67.3% are white women and 32.6% are women of color. People of color are 38% of the U.S. population, which means that women of color are underrepresented in leading roles.
• No gender differences are found in terms of leading or supporting characters who are LGBTQ+.
• No gender differences were found in terms of characters with disabilities.
• With age, a majority of female characters in family films are under age 40 (60.2%), while a majority of male characters are age 40+ (56.4%) in 2019.
• No gender differences are found in terms of characters who are 60+.
• Male characters are more likely than female characters to be presented as large (9.7% compared with 5.8%).

RACE/ETHNICITY

In this section, we examine race and representation in terms of character prominence, stereotypes, work and leadership, traits, and box office returns. For clear analysis, we group race into two categories—characters of color and white characters.

Prominence

• People of color account for 38% of the U.S. population,⁷ and in 2019, a historic 30% of family films featured a person of color as a lead or co-lead.
• Leads of color have made steady progress in family films, with the exception of a major dip in 2011, when leads of color fell to 7%.

GD-IQ: Screen Time

• Characters of color were on screen 42.5% of the time in the top-grossing family films of 2019.

Stereotypes

• White characters are more likely than characters of color to be portrayed as having higher socioeconomic status (“upper-class”) in family films (14.8% compared with 8.7%).
• White characters and characters of color are equally likely to be shown as violent, engaging in criminal behavior, and dying.

White characters are more likely to be shown with a higher socioeconomic status

Work and Leadership

• White characters and characters of color are equally likely to be shown as having an occupation.
• A greater percentage of characters of color are shown as hard working than white characters (56.9% compared with 48.9%).⁸
• White characters are more likely to be shown as leaders than characters of color (46.7% compared with 39.8%).

Character Traits

• No racial differences emerge in terms of intelligence and humor portrayals in family films.

Box Office

• When it comes to domestic box office revenue, family films with leads of color have steadily earned more, peaking in 2016 at $95.6 million.
• Starting in 2016, family films led by people of color have generally grossed more on average than films with white leads, but this trend reversed in 2019. Family films with white leads earned significantly more at the box office than films with leads of color in 2019 ($96.9 million compared with $66.4 million).
• Worldwide box office revenues for family films with leads of color have steadily increased in the past decade, rising from $82 million in 2007 to $182.8 million in 2019.
• A persistent gap exists in average worldwide box office returns from family films with white leads compared with leads of color.

LGBTQ+

In this section, we examine representations of LGBTQ+ characters when it comes to prominence, stereotypes, work and leadership, and character traits.

Prominence

• In the U.S., 4.5% of people identify as LGBTQ+,9 and for most of the last decade, the percentage of family films with an LGBTQ+ lead has been stagnant, averaging less than 1%.
• In 2018, family films achieved equality in representation with 5% of LGBTQ+ leads, but this number fell to 2% in 2019.

Stereotypes

Representations of LGBTQ+ characters in family films uphold stereotypes about over-sexualization, sexual promiscuity, and death:

• LGBTQ+ characters are more likely to die in family films than heterosexual characters (16.7% compared to 11.9%).
• LGBTQ+ characters are more likely than heterosexual characters to be shown in revealing clothing (27.8% compared with 8.8%).
• A greater percentage of LGBTQ+ characters are portrayed as sexually promiscuous than heterosexual characters (5.6% compared with 1.6%).

Family films reinforce the stereotype that LGBTQ+ people are more sexually promiscuous

Work and Leadership

• Heterosexual characters are more likely to be portrayed as hard working than LGBTQ+ characters (51.9% compared with 33.3%).
• No differences emerged between LGBTQ+ characters and heterosexual characters along occupation type, STEM representation, or leadership.

Character Traits

• Heterosexual characters are more likely to be portrayed as smart than LGBTQ+ characters (41.6% compared to 27.8%).
• LGBTQ+ characters are more likely to be presented as funny than heterosexual characters (66.7% compared with 37.7%).

DISABILITY

In this section, we examine representations of characters with disabilities when it comes to prominence, stereotypes, work and leadership, and character traits.

Prominence

• In the U.S., 18.7% of people have a physical or cognitive disability,10 and for most of the last decade, they averaged only about 1% of leads in family films.
• However, in 2018 and 2019, 8% of films featured a lead with a disability—a historic high.

Prominence

People with disabilities are often characterized by two familiar tropes– the “super crip” and the “bitter crip.”¹¹ In both types of portrayals, disability becomes the most prominent aspect of a character, and the storyline revolves around the individual’s relation to their disability. The “bitter crip” is a character who is overcome by their suffering, and she or he often becomes the villain or antagonist. The “super crip” is an individual whose life revolves around heroically and heartwarmingly overcoming their disability, which serves as a central motivator. Although this can initially seem complimentary, these storylines often reinforce the superiority of people without disabilities (and thus, the importance of “overcoming” a disability), glorify those with disabilities for being able to live a normal or successful life, and support the notion that overcoming is a matter of personal character rather than highlighting institutional or structural barriers that can make it more difficult for those living with disability to have the same resources and opportunities.

• One-in-five (22.9%) characters with disabilities in popular family films fit the “Super Crip” stereotype.
• Nearly one-in-ten (8.6%) characters with disabilities are portrayed using the “Bitter Crip” stereotype.
• Characters with disabilities are more likely to be rescued in family films than other characters (34.3% compared with 20.6%).
• Characters with disabilities are nearly twice as likely to die in family films as other characters (20.0% compared with 11.7%).

Characters with disabilities are twice as likely to die in family films than other characters

Work and Leadership

• Characters with disabilities are more likely to be represented as hard working than other characters (65.7% compared with 50.7%).
• Characters with disabilities are twice as likely to be shown in management positions than other characters (20.0% compared with 10.7%).
• Characters with disabilities are more likely to be portrayed in STEM occupations than other characters (20.0% compared to 11.0%).
• Characters with disabilities are more likely to be shown as leaders than other characters (54.3% compared with 44.1%).

Character Traits

• No differences were found in terms of intelligence and character disability.
• Characters with disabilities are more likely to be portrayed as humorous (51.4% compared to 37.5%). No difference is found in terms of rates of physical comedy.

Age (60+)

This section analyzes representations of characters age 60 and older in terms of prominence, stereotypes, work and leadership, and character traits. We are interested in knowing whether older characters are included in family films, and when they appear, whether their representations reinforce or challenge ageist stereotypes.

Prominence

• People 60+ make up 19.0% of the American public, but only 9.1% of characters in family films.
• Only 3.5% of leading characters in 2019 family films are age 60+.
• Only 3.5% of leading characters in 2019 family films are age 60+.

Family films reinforce the stereotype that older people are not physically attractive

Stereotypes

• Characters age 60+ are equally likely to die in family films as younger characters.
• Characters age 60+ are more likely to be portrayed as ugly/physically repulsive than younger characters (5.5% compared with 0.9%). Characters 60+ are three times more likely to be portrayed as “worse than average looking” than younger characters (30.9% compared with 10.8%). This reinforces the stereotype that older people are not physically attractive.
• Characters age 60+ are more likely to be shown with zero sexual partners than younger characters (97.3% compared with 87.2%). This gap reinforces the stereotype that those 60+ are non-sexual, and that intimacy is reserved for young characters that conform to our beauty ideals.

Family films reinforce the stereotype that people 60+ are non-sexual

Work and Leadership

• Only 39.1% of characters ages 60+ are shown as having an occupation compared with 80.7% of younger characters.
• Characters ages 60+ are twice as likely to be shown in a management occupation than younger characters (21.6% compared with 10.1%).
• Elderly characters are equally likely to be shown in STEM professions as younger characters.
• Characters ages 60+ are more likely to be shown as leaders than younger characters (51.8% compared with 43.9%).

Character Traits

• Characters who are 60+ are equally likely to be portrayed as intelligent as younger characters.
• Characters ages 60+ are more likely to be shown as funny than younger characters (40.9% compared with 35.4%).

Body Size (Large Body Type)

Forty percent of American adults are large, defined as having a Body Mass Index of 25 or greater. According to a recent report from the Centers for Disease Control and Prevention, the average man in the U.S. is 5’9” and weighs 196 pounds, while the average woman is just under 5’4” tall and weighs 169 pounds. In this report, we are interested in whether family films present body size diversity, and whether large characters are portrayed in ways that reinforce damaging stereotypes. For purpose of this analysis, we group characters into large (somewhat or very large) and non-large.

Prominence

• Overall, only 8.3% of characters are large, which means this identity group is underrepresented in family films as compared with the broader population.
• When it comes to large leads, 6.8% of leading or co-leading characters are presented as large.

Stereotypes

A sizeable number of large characters are presented as negative stereotypes associated with their size:

• 7.4% of large characters are presented as lazy.
• 8.5% of large characters are portrayed as physically slow.
• 9.6% of large characters are presented as stupid.
• 8.5% of large characters are shown as poorly dressed.
• 7.4% of large characters are portrayed as clumsy.
• One-in-ten (9.6%) large characters are portrayed as a punchline in family films.

One-in-ten large characters are used as a punchline in family films

Stereotypes

• Non-large characters are more likely to have a job/occupation than large characters (34.8% compared with 20.7%).
• Non-large characters are more likely to be shown as hard working than large characters (52.5% compared with 40.4%).
• We find no differences in terms of body size and the types of jobs held or leadership in the workplace.

Character Traits

• Large characters are less likely to be shown as smart than other characters (35.1% compared with 42.1%).
• One-third (33.0%) of large characters are presented as funny. One-in-five (18.1%) large characters are presented as comic relief.
• When it comes to verbal shaming from other characters or narrators, 7.4% of large characters are verbally shamed in family films.

GENDER REPRESENTATIONS:

The fact that the percentage of family films with women leads reached parity in 2019, at 48%, is profoundly significant. Gender parity in leading characters in family films shows great progress, as does the increase in speaking time (from 31.3% in 2014 to just under 40.0%) and screen time (from 34.9% to 42.6%). But despite this progress, women are still vastly underrepresented as supporting characters and minor characters, female characters are more likely than male characters to be shown in revealing clothing, and male characters are more likely to be shown as violent. Content creators are making rapid improvements in including more women in family films, but the ways in which they are represented need improvement.

RACE/ETHNICITY REPRESENTATIONS:

Furthermore, leads of color in family films increased significantly in the past decade, reaching 30% for the first time. The percentage of leads of color has increased dramatically in the past decade to a historic high, but racial stereotypes persist. White characters are more likely than characters of color to be portrayed as having higher socioeconomic status and shown as leaders.

DISABILITY REPRESENTATIONS:

Leads with disabilities have reached historic highs in the last two years (at 8%). Family films are now including more characters with disabilities than in the past, but portrayals reinforce stereotypes. They are more likely to be rescue and die in family films than other characters. On the other hand, characters with disabilities are more likely to be represented as hard working, in management positions, in STEM occupations, and as leaders than other characters.

LGBTQ+ REPRESENTATIONS:

When it comes to LGBTQ+ portrayals, family films reached a historic high in 2018, but declined in 2019. Also, portrayals reinforce stereotypes of LGBTQ+ characters as overly-sexual and sexually promiscuous, and are more likely to die in family films. Additionally, heterosexual characters are shown as smarter and harder working than LGBTQ+ characters.

AGE REPRESENTATIONS:

Characters ages 60+ are underrepresented in family films compared to the broader population, and the ways they are presented tend to reinforce stereotypes that elderly people are non-sexual and not working. On a positive side, characters who are 60+ are more likely to be portrayed as managers and leaders in the workplace.

SIZE REPRESENTATIONS:

Large people are underrepresented in family films compared to the U.S. population. When they are shown, it is too often in ways that reinforce damaging stereotypes of large people as lazy, physically slow, stupid, poorly dressed, clumsy, and as a punchline. Large people are shown as less smart and face verbal shaming.

GEENA DAVIS INSTITUTE RESEARCH TEAM

Dr. Caroline Heldman

Soraya Giaccardi, Rebecca Cooper, Dr. Ian Breckenridge-Jackson, Dr. Meredith Conroy, Dr. Patricia Esparza, Dr. Linzi Juliano, Dr. Ninochka Mctaggart, Dr. Rita Seabrook, Emma Burrows, Sofie Christensen, Nathan Cooper Jones, Romeo Perez, Hannah Phillips, Jenna Virgo

USC VITERBI SAIL-MICA TEAM

Dr. Shrikanth Narayanan

Dr. Naveen Kumar, Dr. Tanaya Guha, Krishna Somandepalli, Anil Ramakrishna, Nikolaos Malandrakis, Victor Martinez, Karan Singla, Rahul Sharma, Rajat Hebbar, Chloe Tanlimco, Digbalay Bose, Rimita Lahiri, Timothy Greer, Sabyasachee Baruah, Ming-Chang Chiu, Jamie Flores, Alex Young

To date, most research investigations of media representations have been done manually. The GD-IQ revolutionizes this approach by using automated analysis, which is not only more precise, but makes it possible for researchers to quickly analyze massive amounts of data, which allows findings to be reported in real time. Additionally, the GD-IQ allows for more accurate analysis, and because the tool is automated, comparisons across data sets and researchers are possible, as is reproducibility. Automated analysis of media content gets around the limitations of human coding. Beyond the significant advantage of being able to efficiently analyze more films in less time, the GD-IQ can also calculate content detail with a level of accuracy that eludes human coders. This is especially true for factors such as screen and speaking time, where near exact precision is possible. Algorithms are a set of rules of calculations that are used in problem-solving. For this report, we employed two automated algorithms that measure screen time by gender and race, and speaking time of characters by their gender. Here is an overview of the procedures we used for each algorithm.

SCREEN TIME ANALYSIS

We compute the screen time of female characters by calculating the ratio of female faces to the total number of faces in the film’s visuals. The screen time is calculated using online face detection and tracking with tools provided by Google’s machine learning technology. In the interest of precision and time, we estimate screen time by computing statistics over face-tracks (boxes tracking the general outline of each face) instead of individual faces. The face-tracks returned by technology include different attributes of the face with the corresponding time of occurrence in the video. Among the attributes returned for each of the detected faces, we use two parameters – the confidence of the detected face and the system’s posterior probability for gender prediction. A threshold of 0.25 was empirically chosen for determining confident face detection.

Due to multiple characters appearing on screen simultaneously, the face-tracks can be overlapping. A gender label is then assigned to each track using the average gender posterior associated with the confident faces in the track. If the average gender posterior probability of the track is greater than 0.5, the track is classified as a “female track,” otherwise, it is a “male track.” The number of frames with confident face detections in each track is summed up across all tracks to get the total number of faces. The number of female tracks is aggregated to get the total number of faces predicted as female. Finally, the screen time is computed as the ratio between the number of female face detections to the total number of face detections across the length of the movie. Supplementary analysis shows that screen time estimated at frame-level (individual faces) instead of using face-tracks was not significantly different and was comparable. Furthermore, computing the average of gender posterior over tracks has an added benefit of “smoothing out” some of the local gender prediction errors. Face-tracking incorporates temporal contiguity information to reduce transient errors in gender prediction that may occur with analyzing individual faces independently. We performed a similar analysis for character race and screen time.

SPEAKING TIME ANALYSIS

Using movie audio, we compute the speaking time of male and female characters to obtain an objective indicator of gender representation. The algorithm for performing this analysis involves automatic voice activity detection, audio segmentation, and gender classification.

VOICE ACTIVITY DETECTION:

Movie audio typically contains many non-speech regions, including sound effects, background music, and silence. The first step is to eliminate non-speech regions from the audio using voice activity detection (VAD) and retain only speech segments. We used a recurrent neural network based VAD algorithm implemented in the open-source toolkit OpenSMILE to isolate speech segments.

SEGMENTATION:

We then break speech segments into smaller sections in order to ensure each segment includes speech from only one speaker. This is performed using an algorithm based on Bayes Information Criterion (BIC), available in the KALDI toolkit. Thirteen dimensional Mel Frequency Cepstral Coefficient (MFCC) features are used for the automatic speaker segmentation. This step essentially decomposes continuous speech segments obtained in the VAD step into smaller segments to make sure no segment contains speech from two different speakers.

GENDER CLASSIFICATION

The speech segment is then classified into two categories based on whether it was likely spoken by a male or female character. This is accomplished with acoustic feature extraction and feature normalization.

ACOUSTIC FEATURE EXTRACTION

We use 13-dimensional MFCC features for gender classification because they can be reliably extracted from movie audio, unlike pitch or other high-level features where extraction is made unreliable by the diverse and noisy nature of movie audio.

FEATURE NORMALIZATION

Feature normalization is deemed necessary to address the issue of variability of speech across different movies and speakers, and to reduce the effect of noise present in the audio channel. Cepstral Mean Normalization (CMN) is a standard technique popular in Automatic Speech Recognition (ASR) and other speech technology applications. Using this method, the cepstral coefficients are linearly transformed to have the same segmental statistics (zero mean).Classification of the speaker as either male or female is based on gender-specific Gaussian mixture models (GMMs) of the acoustic features. These models are trained on a gender-annotated subset of general speech databases used for developing speech technologies using frame-level features for each gender. The GMM we use in this system has 100 mixture components and is optimized by tuning the parameters in a held-out evaluation set.  For a new input segment whose gender label is to be predicted, the likelihoods of the segment belonging to a male or female class are computed based on this pre-trained model. The class with higher likelihood is assigned to the segment as the estimated gender prediction. The total speaking time by gender is then computed by adding together the durations for each utterance classified as Male/Female. This gives us the male and female speaking time in a movie.

HOW TO CITE THIS STUDY

Heldman, Caroline, Rebecca Cooper, Shrikanth Narayanan, Krishna Somandepalli, Emma Burrows, Sofie Christensen, Nathan Cooper-Jones, Meredith Conroy, Soraya Giaccardi, Linzi Juliano, Ninochka McTaggart, Romeo Perez, Hannah Phillips, Rita Seabrook, and Jenna Virgo (2020). See Jane 2020 Report. The Geena Davis Institute for Gender in Media at Mount St. Mary’s.