Melinda Gates and The Bias In Surveys

Having a huge amount of data can be quite different from having enough of the right data. 

Each year Bill and Melinda Gates write an open letter about the issues and innovations most striking to them as part of their philanthropic activities. 

In an article titled “5 Global Health Issues That Surprised Bill and Melinda Gates Last Year”, Time magazine reported on their 2019 letter as follows: 

For Melinda, the most surprising of the nine issues they highlight is the fact that data is still sexist, and critical information on the state of women’s health is missing. “When I got into this work, we couldn’t make investments without good data. We think data is objective, but it’s not. If we don’t collect data about women, about women’s lives, and if we don’t ask the right questions, we inadvertently bias the data”. 

One of the recently quoted examples of bias is in the development of new medical drugs. 

Michelle Llamas writing for the web site drugwatch in an article that features 22 Cited Research Articles noted, in part, the following: 

According to the FDA’s Office of Women’s Health, females have nearly double the risk of developing an adverse drug reaction compared to men. 

Yet, for decades, women were underrepresented in clinical trials, and the FDA didn’t start looking into the problem of gender equality in studies until the late 1990s. In fact, until 1988, clinical trials of new drugs were conducted on predominantly male subjects, though women consumed 80 percent of the pharmaceuticals in the U.S. at the time. 

Apparently, this issue goes deeper than just the final trials on humans. In general, researchers prefer male animals because male physiology is easier to study. 

Caroline Criado Perez in the preface to her book Invisible Women says: “Numbers, technology, algorithm's, all of these are critical to the story of invisible women. But they only tell half the story. Data is just another word for information and information has many sources. Statistics are a kind of information. Yes, but so is human experience”. 

Our perception of desired transport systems is largely based on measuring past and existing activities. But if our measurements reflect the trends and relationships of an inequitable environment, then we will perpetuate that inequality into the future.  

How often do we test if a survey is gender-biased?  

How often do we say “transport is a derived activity”? Consequently, how often should we say “who has and who doesn’t have the opportunity to access transport”? 

COVID-19 is showing us that the future could be significantly different from the past and so we have to understand the social context not only to serve needs fairly but to predict the changing nature of the task into the future. 

It is not enough to just have good intentions; do we regularly (always?) test data and analysis for whether it has a gender or other biases? 

If you ask the driver of a car (which is more likely to be a man), what the purpose of the trip is, human nature suggests they will usually state their needs, but many trips are multipurpose: are women and children being adequately represented? Indeed, major survey data are often categorised into the “main” purpose thus deleting the other aspects of the trips. If someone drives to work and drops their partner off at a closer point of employment or the shops or children off at school, what really is the “main” purpose. Even if you have a priority, we are unlikely to save and analyse the data for what it shows us about the other purposes? 

It’s worth repeating a quote from Melinda Gates “When I got into this work, we couldn’t make investments without good data. We think data is objective, but it’s not.” 

Will it be enough to encourage people to do this to the depth it requires? 

To further quote Llamas: “Studies conducted by the NIH in the years following its much-lauded 1993 Act (the National Institutes of Health Revitalization Act mandated the inclusion of women in clinical trials) did include more women. Yet, about 75 percent of studies published in four major journals in 1993, 1995, 1997 and 1998 failed to analyse data by gender”.