The Wall Street Journal - 21.10.2019

THE WALL STREET JOURNAL. Monday, October 21, 2019 |A19

Blind Spots in the ‘Blind Audition’ Study

sizes, contradictory results and failures to pass standard tests of statistical significance. But few readers seem to have noticed. What caught everyone’s attention was a big claim in the final paragraph: “We find that the screen increases—by 50 percent—the proba- bility that a woman will be ad- vanced from certain preliminary rounds and increases by severalfold the likelihood that a woman will be selected in the final round.” According to Google, the study has received more than 1,500 cita- tions in academic articles and thou- sands of media mentions. It has been featured in TED Talks, cele- brated at the Davos conference, and showcased in so many diversity workshops that one attendee begged never to hear about it again. In- spired by the “academically verified Orchestra study,” GapJumpers, a Sil- icon Valley startup, offers compa- nies software to conduct blind inter- views in other contexts. The study’s appeal is clear: Two prominent economists, in a top journal, wielding state-of-the-art econometrics, captured and quan- tified bias against women and documented a solution. Or so it seemed. The research went uncriticized for nearly two decades. That changed recently, when a few schol- ars and data scientists went back and read the whole study. The first thing they noticed is that the raw tabulations showed women doing worse behind the screens. But perhaps, Ms. Goldin and Ms. Rouse ex- plained, blind auditions “lowered the average quality of female audi- tionees.” To control for ability, they analyzed a small subset of candidates who took part in both blind and nonblind auditions in three of the eight orchestras.

The result was a tangle of am- biguous, contradictory trends. The screens seemed to help women in preliminary audition rounds but men in semifinal rounds. None of the findings were strong enough to draw broad conclusions one way or the other. So where did Ms. Goldin and Ms. Rouse get their totemic conclusion that blind auditions dramatically improved the success of women candidates? After warning that their findings were not statistically significant, they declared them to be “economically significant.” What does that mean in this context? “That doesn’t mean anything at all,” writes Columbia University data scientist Andrew Gelman, in a recent post about the study. “Some fine words but the punchline seems to be that the data are too noisy to form any strong conclusions.” My guess is that the authors thought they had detected something with real-world relevance despite an absence of statistical rigor. But that’s a reason to call for more research, not to declare the transformative power of screens in women’s quest for equality.

Still, isn’t it obvious that the screens at least contributed to equal hiring? No. The screens might have been a reflection of changing attitudes, and perhaps those attitudes, not the screens, helped women. After all, women didn’t need blind auditions to move ahead in law, business, medicine or the academy—or at the Cleveland Or- chestra, which, according to the study, did not use them. Mr. Gelman and the other critics don’t deny the existence of bias against women or question the po- tential merits of blind auditions. Anonymous tryouts make sense as a means of achieving impartiality. But Ms. Goldin and Ms. Rouse verified nothing about the value of blind recruitment for women. Nor has anyone else. The subsequent research is a morass of base- less claims, retracted statements and contradictory findings. There is, however, one study that stands out for its rigor and transparency. In 2017 a team of behavioral economists in the Australian government published the results of a large, ran- domized controlled study entitled

“Going Blind to See More Clearly.” It was directly inspired by the blind-audition study. Iris Bohnet, a Harvard Kennedy School dean and Goldin-Rouse enthusiast, served as an adviser. For the study, more than 2,000 managers in the Australian Public Service were asked to select re- cruits from randomly assigned résu- més—some disguising the appli- cant’s sex, others not. The research team fully expected to find far more female candidates shortlisted when sex was disguised. But, as the stunned team leader told the local media: “We found the opposite, that de-identifying candidates reduced the likelihood of women being selected for the shortlist.” It turned out that many senior managers, aware that sexist assumptions had once kept women out of upper-level positions, already practiced a mild form of affirmative action. Anony- mized hiring was not only time-con- suming and costly, it proved to be an obstacle to women’s equality. The team plans to look elsewhere for solutions. Truth matters. Overhyped claims of scientific certainty create confu- sion, undermine public trust and send scarce resources in the wrong direction. Most of all, they don’t solve problems. Sex discrimination in the workplace is a serious mat- ter. But improvements require solid data, replicable research and care- ful evaluations of causation. As the scholar Alice Dreger says, “Evi- dence is an ethical issue.” If blind auditions aren’t helpful to women, it’s important we know.

Ms. Sommers is a resident scholar at the American Enterprise Institute and author of “Who Stole Feminism?” and “The War Against Boys.”

By Christina Hoff Sommers

ISTOCK/GETTY IMAGES

I

t is one of the most famous
social-science papers of all
time. Carried out in the
1990s, the “blind audition”
study attempted to document
sexist bias in orchestra hiring. Lion-
ized by Malcolm Gladwell, extolled
by Harvard thought leaders, and
even cited in a dissent by Justice
Ruth Bader Ginsburg, the study
showed that when orchestras audi-
tioned musicians “blindly,” behind a
screen, women’s success rates
soared. Or did they?

Nobody questions the basic facts
that led to the study’s publication.
During the 1970s and ’80s, Amer-
ica’s orchestras became more open
and democratic. To ensure impar-
tiality, several introduced blind au-
ditions. Two economists, Claudia
Goldin of Harvard and Cecilia
Rouse of Princeton, noticed that
women’s success rates in auditions
increased along with the adoption
of screens. Was it a coincidence or
the result of the screens? That is
the question the two economists
tried to answer in “Orchestrating
Impartiality: The Impact of ‘Blind’
Auditions on Female Musicians,”
published in 2000 in the American
Economic Review.
They collected four decades of
data from eight leading American
orchestras. But the data were in-
conclusive: The paper includes mul-
tiple warnings about small sample

A lauded 2000 article

claiming to find sexism in

American orchestras looks

increasingly spurious.

As America Leaves Syria, Iran Isn’t as Happy as You Think

I

t is the conceit of the commen-
tariat that Iran is a winner of
the latest mayhem in the Middle
East—the departure of U.S. troops
from Syria and the subsequent
Turkish incursion. Yet the clerical
oligarchs seem anxious about all
that is happening around them.
A continuing Syrian civil war was
working for Tehran. It had managed
to navigate skillfully the politics at
play, developing good relations with
both Bashar Assad and the Kurdish
militias opposing him. The latter’s
U.S. support created a sort of bal-
ance between the sides. Despite his
precarious situation, Mr. Assad
dreams of unifying Syria. His Ira-
nian patrons have long advised him
to limit his ambitions and consoli-
date power in the territory he com-
mands. With the U.S. backing the
Kurds, Mr. Assad had to follow
Iran’s advice or risk a wider war he
could ill afford.
Now that the U.S. is gone and the
militias have been forced to ally
with Mr. Assad, little stops him
from trying to seize the 40% of the

country he doesn’t control. After nearly a decade of war, Syria is ex- hausted. Any attempt to control more land is bound to jeopardize Mr. Assad’s existing gains. If Mr. Assad is mired in a longer and cost- lier war, Tehran would have to commit even greater resources to this conflict at a time when its economy is suffering due to sanctions and mismanagement. Turkey looms large in the clerical imagination. In a region littered with weak and failing states, Turkey is a formidable power with its own ambitions for the Middle East. Iran is already at loggerheads with Saudi Arabia and caught in a costly sec- tarian conflict across the region. Ankara’s intervention in Syria is bound to complicate Iran’s designs in the Levant. Iran had forged de- cent ties with Kurdish forces and it viewed its buffer zone in northern Syria as a useful check on Turkey. With the U.S. gone, the Kurds have been swept back before the Turkish advance. Iranian leaders including Presi- dent Hassan Rouhani and Foreign Minister Javad Zarif urged Turkey

not to move in to Syria. Ayatollah Ahmad Khatami, the hard-line Fri- day prayer leader, captured the es- sence of Iran’s concerns, warning Turkey “not to stumble into Amer- ica’s trap” and thus find itself in a quagmire similar to the Saudi pre- dicament in Yemen. In a nonverbal

warning, Iran conducted military maneuvers along its border with Turkey shortly before Ankara in- vaded Syria, according to Reuters. With the U.S. gone, Iran’s at- tempts at persuasion will do little to restrain Turkey. In Syria, Ankara is bound to be more powerful, Mr. Assad more reckless, the Kurds sub- stantially weakened, and Islamic State reconstituted. None of this is good news for Iran, which hoped to make incremental gains in the Le-

vant by keeping the Syrian conflict simmering at a low burn. And the current flare-up comes at a bad time for Tehran. In one of the many paradoxes of the Middle East, Iran gained much from the U.S. top- pling Saddam Hussein. Iraq’s Sunni strongmen have haunted Persian monarchs and mullahs for decades, even launching an invasion in 1980 that traumatized a generation of Iranians. Since the 2003 invasion, Iran-backed Shiite militias have op- erated as an auxiliary force, coming to dominate Iraq—one of the Islamic Republic’s most consequential suc- cesses. Iran uses Iraq to transport its oil to the global markets in defi- ance of American sanctions. But Iran’s domination doesn’t go over well. Tehran is the key actor in selecting Iraq’s prime ministers and Parliaments. This means it gets the blame for the Iraqi government’s corruption and inefficiency. Iraqi resentment of Iran exploded into protests this month that shook the foundations of the Baghdad government. Tehran has tried to smooth things over. Iranians repeat the con-

spiracy theory that the U.S. and Saudi Arabia instigated Iraqi protests. Supreme Leader Ali Khamenei warned that “Iran and Iraq are two nations whose hearts and souls are tied together....Enemies seek to sow discord, but they’ve failed, and their conspiracy won’t be effective.” Mr. Khamenei’s putative successor, Ayatollah Ebrahim Raisi, reinforced his narrative, stressing that sedition is taking hold of Iraq. Beneath the stale rhetoric, the mullahs likely re- alize that the Iraqi government’s hold on power is tenuous. If it col- lapses, there could follow another messy civil war whose outcome Teh- ran may not be able to condition. On top of all of that, now Syria is in flames. There are many sound ar- guments about why the U.S. should not have withdrawn its modest presence. But the notion that the pullout empowers Tehran is belied by its leaders’ expressions of anxi- ety. The Middle East rarely offers a respite to ambitious nations, even Iran.

Mr. Takeyh is a senior fellow at the Council on Foreign Relations.

By Ray Takeyh

Tehran finds itself at cross-purposes with Damascus and Ankara as Baghdad slips away.

OPINION

PUBLISHED SINCE 1889 BY DOW JONES & COMPANY Rupert Murdoch Executive Chairman, News Corp Matt Murray Editor in Chief

Robert Thomson Chief Executive Officer, News Corp William Lewis Chief Executive Officer and Publisher

EDITORIAL AND CORPORATE HEADQUARTERS:
1211 Avenue of the Americas, New York, N.Y., 10036
Telephone 1-800-DOWJONES

DOW JONES MANAGEMENT: Ramin Beheshti, Chief Technology Officer; Natalie Cerny, Chief Communications Officer; Kamilah Mitchell-Thomas, Chief People Officer; Edward Roussel, Chief Innovation Officer; Christina Van Tassell, Chief Financial Officer OPERATING EXECUTIVES: Kenneth Breen, Commercial; Jason P. Conti, General Counsel; Tracy Corrigan, Chief Strategy Officer; Frank Filippo, Print Products & Services; Kristin Heitmann, Chief Commercial Officer; Nancy McNeill, Corporate Sales; Thomas San Filippo, Customer Service; Josh Stinchcomb, Advertising Sales; Suzi Watford, Chief Marketing Officer; Jonathan Wright, International Barron’s Group: Almar Latour, Publisher Professional Information Business: Christopher Lloyd, Head; Ingrid Verschuren, Deputy Head

Neal Lipschutz Karen Miller Pensiero
Deputy Editor in Chief Managing Editor
Jason Anders, Chief News Editor;
Thorold Barker, Europe; Elena Cherney, Coverage
Planning; Andrew Dowell, Asia; Alex Martin, Print
& Writing; Michael W. Miller, Features & Weekend;
Emma Moody, Standards; Shazna Nessa, Visuals;
Matthew Rose, Enterprise; Michael Siconolfi,
Investigations; Louise Story, Strategy and Interim
Product & Technology; Nikki Waller, Live
Journalism; Stephen Wisnefski, Professional News

Gerard Baker, Editor at Large

Paul A. Gigot, Editor of the Editorial Page;
Daniel Henninger, Deputy Editor, Editorial Page

WALL STREET JOURNAL MANAGEMENT:
Joseph B. Vincent, Operations;
Larry L. Hoffman, Production

Open Skies Help Keep the Peace With Russia

I

nternational security isn’t a
given. Historically, peace among
the great powers is a rarity. It’s
also a great accomplishment. Like
trust, peace and security take a long
time to build and only a moment to
dismantle.
One of the pillars upholding in-
ternational peace and security today
is the 1992 Open Skies Treaty.
Thirty-four nations, including the
U.S. and Russia, have agreed to this
treaty, which allows signatories to
fly unarmed surveillance aircraft
over one another’s territory. This
important tool, known as overflight,
has been especially useful for the
U.S. and our allies to monitor Rus-
sian military activities. Even when

relations between Moscow and Washington are tense, the Open Skies Treaty helps preserve a mea- sure of transparency and trust. This great accomplishment of post-Cold War diplomacy could soon be erased if, as has been widely re- ported, some Trump administration officials have their way and the U.S. unilaterally exits the treaty. Such a withdrawal would be a grave mis- take. It would undermine trust between the U.S. and Russia and en- danger American allies. Since the emergence of the super- power nuclear-arms race, leaders in Moscow and Washington have sought to avoid all-out war. They’ve had to overcome mutual distrust and negotiate agreements to manage military competition, reduce tensions, and lower the risk of surprise attack.

The idea for the Open Skies Treaty dates back to the 1950s. Pres- ident Dwight D. Eisenhower realized that without better information about each side’s capabilities, worst- case assumptions would drive deci- sions and exacerbate risks. In 1955 he made a bold proposal: The U.S. would permit unarmed Soviet aircraft to make unlimited surveillance flights over U.S. territory if the So- viet Union would reciprocate. U.S. allies, the American public and many congressional leaders backed the idea, but the Soviets were skeptical and the proposal was shelved. The two sides went on to negotiate a se- ries of nuclear arms-control agreements, beginning in the 1960s, that verifiably capped and later slashed their enormous nuclear arsenals. As the Cold War ended, Presi- dent George H.W. Bush outlined a

new vision for security that in- cluded a fortified version of Open Skies. Bush saw overflight as an effective way to verify the new limits on military forces established by

1990’s Conventional Forces in Eu- rope agreement. The idea was sup- ported by smaller European countries that believed it would be beneficial to have an independent ability to monitor events around the continent. Moscow hesitated at first, but in 1992 the new Russian government agreed to open its entire territory to observation and overflight. The Open Skies Treaty was signed in Hel- sinki in 1992 and took effect a decade later. The treaty has authorized more than 1,426 missions, including more than 500 U.S. flights over Russia, which is by far the most overflown and best-monitored country in the treaty. The flights, scheduled on short notice, provide valuable photographic evidence of major military move- ments across Europe, reducing uncer- tainty and worries about surprise attack. They add important information to what satellites provide. The treaty stipulates that mission aircraft can be equipped only with specified sensors limited to an agreed resolution. By agreement of all parties, including the U.S., a pro- cess is under way to upgrade the sensors. These detailed, verifiable procedures allow observing parties

to identify significant military equip- ment, such as artillery, fighter aircraft and armored combat vehicles. All imagery collected from flights is made available to any signatory. As with any treaty, implementa- tion disputes arise. Current dis- agreements are related to underlying territorial and political issues between Russia and some of its neigh- bors. But these problems can be solved through professional, prag- matic diplomacy, not by abandoning treaty commitments. Today, Republicans and Democrats agree that Vladimir Putin’s Russia poses serious international-security challenges. Rather than walk away from security agreements that help the U.S. and its allies manage the risks posed by Moscow, Washington needs to redouble its longstanding commit- ment to proven risk-reduction strate- gies and arms-control treaties ad- vanced by successive presidential administrations. Unilateral withdrawal from Open Skies would damage the security of the U.S. and its allies. We respectfully urge President Trump to reject calls to abandon the treaty. Congress also needs to ap- prove Pentagon requests for up- grades to U.S. observation aircraft, as other Open Skies countries, like Germany, are already doing. Open Skies has become what Eisenhower envisioned—a critical confidence-building treaty that im- proves Euro-Atlantic security with every flight. The U.S. should preserve this agreement, particularly in a time of renewed tensions with Russia.

Mr. Shultz served as secretary of state, 1982-89. Mr. Perry served as defense secretary, 1994-97. Mr. Nunn, a Democrat, was a U.S. senator from Georgia, 1972-97, and was chairman of the Armed Services Committee.

By George P. Shultz,
William J. Perry
And Sam Nunn

Ike’s idea, codified in a 1992 treaty, is still a good one. The U.S. shouldn’t abandon the pact.

From James Freeman’s Best of the Web, WSJ.com, Oct. 18:

Mrs. Clinton also offers a history lesson. She notes that the current impeachment effort will be more challenging than the Watergate in- quiry she worked on as a young congressional aide, in part because of all the new media competition. Back in Nixon days there were just three commercial broadcast televi- sion networks, plus dominant news- papers. Now, she says, “I think it’s a lot harder for Americans to know what they’re supposed to believe.” She tells the story of dealing with questions from reporter Sam Donald- son in the 1970s, rather than today’s myriad online media competitors. “It was a much more controllable envi- ronment,” she laments.

Notable Quotable

The Wall Street Journal - 21.10.2019

THE WALL STREET JOURNAL. Monday, October 21, 2019 |A19

Get our desktop app

Company

Features

Documentation

Resources