Peering over the shoulders of recruiters: Hiring discrimination on a recruitment platform

Here’s an exciting new study on hiring discrimination. They got access to the behavioural data of online recruiters to find evidence of discrimination against atypical candidates: Contact rates by recruiters are 4–19% lower for individuals from immigrant and minority ethnic groups, depending on their country of origin, than for citizens from the majority group. Women experience a penalty of 7% in professions that are dominated by men, and the opposite pattern emerges for men in professions that are dominated by women.

I find it interesting that they pitch their method as an alternative to correspondence tests (perhaps not all that novel if we’re looking outside the strict focus on hiring discrimination). We’re seeing an increasing number of correspondence tests in recent years, despite important ethical concerns. Not all of them are reasonably motivated, in my view — “no recent correspondence test” in a particular country/for a particular group/occupation does not cut it for me –, but jointly these studies give us a pretty clear picture of discrimination (especially in Western countries). Access to recruiting databases may not be possible in all countries, and we’re still struggling with the blatant omission of informal labour markets and internal recruitment. On the other hand, at least in principle we could test different interfaces and see if we can reduce discrimination this way…

Correspondence Test Shows Discrimination by School Administrators

A new correspondence study from Denmark shows discrimination by school administrators against parents with ‘Muslim’ names. They sent letters to schools across the country to ask whether they could move their son to that particular school (implying that they were not happy with the current school). 25% of fathers with a ‘Danish’ name (i.e. Peter Nielsen) received a positive answer, compared with 15% of fathers with a ‘Muslim’ name (i.e. Mohammad Osman).

In addition to holding everything constant by using men only (fathers enquiring about their sons), they had a variation in whether the son was a ‘diligent’ student. An interesting qualitative detail is that ‘Muslims’ are more often subjected to additional questions by e-mail (simple questions like verifying they actually live in the catchment area), while the ‘Danes’ were more often asked to call.

I find it interesting that their point of reference were studies on discrimination by public officials (typically politicians), but did not reflect methodological innovations from other correspondence tests, like stimulus sampling (!), or considerations of unmatched designs. I find it disappointing to find that the pre-registration at EGAP leads to a “page not found” error, especially since footnote 1 contains this interesting teaser: “We diverge from the preregistration to limit our focus only to the two variables that were subject to experimental manipulation and causal inference rather than those conditional on posttreatment responses.”

Continue reading “Correspondence Test Shows Discrimination by School Administrators”

How well do correspondence tests measure discrimination?

Correspondence tests are a useful field experiement to measure discrimination in the formal labour market. These correspondence tests are also known as CV experiements: Researchers send two equivalent applications to an employer, differening only in the quantity of interest — gender and ethnicity are common. If only the majority or male candidate is invited for a job interview, we probably have a case of discrimination. Once we aggreate across many employers, we’re pretty confident to have captured discrimination.

Most studies stop there, declining any offer to reduce the burden on employers. The hiring process, however, does not end there. Lincoln Quillian and his team have now compiled a list of studies that went further. They find that the first stage of screening is far from the end of discrimination, and the job interview can increase overall discrimination substantially. Correspondece tests focusing on the first stage will capture only some of the discrimination. Interestingly the discrimination at the job interview stage appears unrelated to discrimination at the first screening of applications.

Quillian, L., Lee, J., & Oliver, M. (2018). Meta-Analysis of Field Experiments Shows Significantly More Racial Discrimination in Job Offers than in Callbacks. Northwestern Workin Paper Series, 18(28). Retrieved from https://www.ipr.northwestern.edu/publications/papers/2018/wp-18-28.html

Zschirnt, E., & Ruedin, D. (2016). Ethnic discrimination in hiring decisions: A meta-analysis of correspondence tests 1990–2015. Journal of Ethnic and Migration Studies, 42(7), 1115–1134. https://doi.org/10.1080/1369183X.2015.1133279

Image: CC-by Richard Eriksson.

Ethnic discrimination in hiring: UK edition

The BBC report on a large correspondent test in the UK carried out by the excellent GEMM project. It’s good to see this reach a wider audience; it’s sad to see the results from our meta-analysis confirmed once again.

British citizens from ethnic minority backgrounds have to send, on average, 60% more job applications to get a positive response from employers compared to their white counterparts

What I really like about this short report by the BBC is that the essentials are covered. Yes we see discrimination, but no, it’s not so bad that none of the minority applicants would ever succeed. They also start the piece with an example of someone changing their name on the CV as a strategy to counter expected (or experienced) discrimination — and they highlight that discrimination has not declined despite policy changes, and indeed that discrimination affects native citizens who happen to have a ‘foreign’ name: they pay for an action of their parents or grandparents.

Are employers in Britain discriminating against ethnic minorities?, GEMM project: PDF of report

Zschirnt, Eva, and Didier Ruedin. 2016. ‘Ethnic Discrimination in Hiring Decisions: A Meta-Analysis of Correspondence Tests 1990–2015’. Journal of Ethnic and Migration Studies 42 (7): 1115–34. https://doi.org/10.1080/1369183X.2015.1133279.

Audit Studies — The Book

There’s a new book edited by S. Michael Gaddis on audit studies. The subtitle promises to go behind the scenes with theory, method, and nuance — and this is what the book provides. As such, the book is a much needed contribution to the literature, where we typically see the results and little how we got there. With (not so) recent concerns around researcher degrees of freedom, the tour behind the scenes offered by the various chapters are an excellent way to make visible and apparent the ‘undisclosed flexibility’ as Simmons et al. called it in 2011. It’s one thing to discuss this in abstract terms, and it’s another thing to sit down with actual research and reflect on the many choices we have as researchers. Indeed, public reflection on research practices may be relatively rare in itself when it comes to quantitative research.

The book comes with a dedicated support webpage: http://auditstudies.com/ (do me the favour to update the “coming soon” banner). On this website, several chapters can be downloaded as pre-prints, though it’s not all the contents if someone is looking for a free book. I hope the authors will make their code available on the website as promised in several places in the book, because this will be another greatly helpful resource for those new to audit studies or looking for new directions.

I greatly enjoyed to read the reflections by other researchers doing audit studies, and would definitely recommend the book to anyone thinking of doing an audit study. At times there were passages that seemed a bit redundant to me, but all the chapters are written in such an accessible way that this didn’t bother me much. Where I think the book falls a bit short is on two fronts. First, it is very US-centric. In itself this is not an issue, but there are several instances where the authors don’t reflect that perhaps in other countries the markets are not organized the same way. In my view, a comparison to other countries and continents would have been fruitful to underline some of these assumptions — I’ve tried to just this on attitudes to immigrants. Second, the book is not a guidebook. I know, it doesn’t claim to be one, but the book asks so many (justified) questions and offers comparatively few concrete guidelines like Vuolo et al. offer it on statistical power. In this sense, the book will stimulate readers to think about their own research design and not provide a template. And this is actually a good thing, because as the chapters make apparent without normally saying so, there is no universal approach that suits different markets in different places and at different times.

So, should you buy the book? Yes if you want to carry out your own audit study, yes if you want to better understand and qualify the results of audit studies, and yes if you’re looking for guidelines — because the book will make you realize that you’re largely on your own. What would probably useful, though, would be a checklist of things to consider, something readers will have to create themselves on the basis of chapters 4 (Joanna Lahey and Ryan Beasley), 5 (Charles Crabtree), and 6 (Mike Vuolo, Christopher Uggen, and Sarah Lageson).

Gaddis, S. Michael, ed. 2018. Audit Studies: Behind the Scenes with Theory, Method, and Nuance. Methodos 14. New York: Springer. https://www.springer.com/cn/book/9783319711522

Ruedin, Didier. 2018. ‘Attitudes to Immigrants in South Africa: Personality and Vulnerability’. Journal of Ethnic and Migration Studies. https://doi.org/10.1080/1369183X.2018.1428086.

Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2011. ‘False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant’. Psychological Science 22 (11): 1359–66. https://doi.org/10.1177/0956797611417632.

Vuolo, Mike, Christopher Uggen, and Sarah Lageson. 2016. ‘Statistical Power in Experimental Audit Studies: Cautions and Calculations for Matched Tests With Nominal Outcomes’. Sociological Methods & Research, 1–44. https://doi.org/10.1177/0049124115570066.

Zschirnt, Eva, and Didier Ruedin. 2016. ‘Ethnic Discrimination in Hiring Decisions: A Meta-Analysis of Correspondence Tests 1990–2015’. Journal of Ethnic and Migration Studies 42 (7): 1115–34. https://doi.org/10.1080/1369183X.2015.1133279.>/small>