Stars don't see gender, and now, NASA is working to not see it either when allocating telescope time to scientists, inspired by a successful experiment with the Hubble Space Telescope.
That experiment tested the hypothesis that if proposals are evaluated without knowledge of who wrote them and strictly on the merit of the science they proposed to do, the astronomers who received highly coveted observing time would end up being a more diverse group. That's the principle behind dual-anonymous review, in which reviewers don't know who submitted which proposals and proposers don't know who reviewed their submissions. Dual-anonymous review is an attempt to reduce the warping power of implicit bias in the traditional review system, in which reviewers are anonymous but proposals include scientists' names.
"We have noticed that in many of our proposal selections, there appears to be a bias in favor of one gender of proposer over another," NASA Astrophysics Division Director Paul Hertz said during a town hall conversation held last month at the 235th meeting of the American Astronomical Society, in Honolulu.
Related: Women in space: A gallery of firsts
By that, he didn't just mean that male scientists receive more than half the opportunities on offer; such a result could simply reflect a greater number of men in the pool of submitters. What Hertz meant is that male scientists generally have a higher success rate in their proposals than female scientists.
The Space Telescope Science Institute, which manages science operations for the Hubble Space Telescope and will do so for NASA's James Webb Space Telescope, saw that trend as well when it studied 20 years' worth of proposals and awards. "That was observable data. You could see it," Heidi Hammel, a planetary astronomer at the Space Telescope Science Institute, told Space.com.
But because the institute had data only about who submitted for telescope time and who received it, there was no way to tell whether the problem came from a difference in proposal quality or from bias, whether conscious or implicit. Hence, the experiment.
The committee reviewing proposals tried a few different ways of shifting the evaluation focus from individual scientists to the science they proposed. For example, one method required that proposals list only scientists' first initials, not their first names. Another required that all the team members be listed in alphabetical order, so it was unclear who was leading the proposal.
Then, for proposals submitted in 2018, the committee went all in, requiring materials that entirely hid the identity of the writers. (A separate document available late in the review process could include nonanonymous information that would speak to the team's ability to conduct the science they proposed to do.)
Related: 'Hidden Figures' explores NASA and civil rights history
According to a report from the committee, scientists submitted nearly 500 proposals that cycle, more than 10 times the number that Hubble had time to gather observations for. Of those proposals, 28% were led by female scientists; of the 40 successful proposals, 12 were led by female scientists. That put female scientists at an 8.7% rate of success and male scientists at 8%, about the same; the year before, without fully anonymous proposals, those numbers were 13% and 24%. "It would be premature to draw broad conclusions, but the results are encouraging" that the anonymous method worked, the statement concluded.
Throughout the experimental process, the committee was assisted and observed by Stefanie Johnson, who specializes in organizational leadership and information analytics at the University of Colorado Boulder's business school. She and a colleague have now published their analysis of the 2018 experiment in the journal Publications of the Astronomical Society of the Pacific.
In that paper, the authors compared dual-anonymous review with other potential tactics for reducing unconscious gender bias, which they described as losing efficacy over time or leading to "backlash against women because women are perceived as receiving extra advantages." (Neither the Hubble committee statement nor Johnson's paper addressed the fact that gender is more complex than a male-female binary; it's unclear whether gender identities were assigned by outsiders or provided by the scientists in question in the Hubble experiment analysis.)
According to the analysis, the switch to dual-anonymous reviewing didn't affect male scientists' success rates to a statistically significant degree but did do so for female scientists. "It's not proof that women will always do better, but hopefully the gender balance will be closer than in years past," Johnson said in a statement. "What this shows is that taking gender out of the equation does allow women to perform better."
Related: Meet the unknown female mathematician whose calculations helped discover Pluto
Although the trial analysis focuses on gender disparities, the idea is that dual-anonymous review should have a similar relationship with a range of different types of potential bias, conscious or implicit. "By going to the dual-anonymous reviews, it isn't only addressing gender issues, it's also addressing underrepresented populations in many different classes," Hammel said, mentioning specifically the draw of a prestigious university or well-respected advisor. "There are many kinds of biases that come into play when you know the names."
NASA looked at the results of the experiment and decided it wanted to give the process a try more broadly. "The proposals were reviewed based on science merit only and not on how familiar the proposer's name was to the reviewers," Hertz said. "It worked great, so we have made the decision to move all of our [General Observer] programs to dual anonymous." The announcement was met with applause from the gathered astronomers in Honolulu.
Hertz's presentation included a slide outlining six observing proposal cycles with deadlines this year that would take the new format, including the first cycle of proposals for time on the James Webb Space Telescope, currently scheduled to launch in March 2021. Hertz said that NASA's Science Mission Directorate, of which his division is a part, is implementing and exploring pilot programs for dual-anonymous review in contexts beyond the General Observer programs as well.
The new policy does mean that scientists will need to learn a new way to write the explanations they submit of why their work deserves an instrument's precious observing time. They can no longer reference their own previous research, partnerships they've built, or funding or observing time they have already won. To smooth the transition, Hertz said, NASA is offering webinars and setting up a help line that scientists could call with questions about anonymizing their research.
"I think it's a good thing. I think it's the right thing. And I'm very pleased to see that NASA is moving forward on transforming their systems to allow these kinds of processes to work for NASA proposals," Hammel said. "Maybe we'll get to the point where people understand [that] in all these cases, you don't need to know the person. You evaluate on the science that they say they're going to do and rank it on that merit."
- Woman, scientist … activist: Female researchers take charge (op-ed)
- Harvard's 'computers': The women who measured the stars
- Margaret Burbidge at 100: The trailblazing astronomer who wouldn't take 'no women' for an answer
Email Meghan Bartels at mbartels@space.com or follow her @meghanbartels. Follow us on Twitter @Spacedotcom and on Facebook.
Join our Space Forums to keep talking space on the latest missions, night sky and more! And if you have a news tip, correction or comment, let us know at: community@space.com.
