meta-analysis – Didier Ruedin

20 October 2022

Does an Immigrant-Native Wage Gap Exist? And Is It Fair?

Blog post by Eva Van Belle over at the NCCR on the move on our meta-analysis and experiment on immigrant-native wage gaps:

People with a migration background face many barriers in the labour market. Research has shown that they have fewer chances of being invited for job interviews or being offered a promotion, and when they do get a job, they may well earn less or face worse working conditions. Much less is known about other forms of labour market discrimination, however. It seems therefore interesting to investigate whether immigrants face wage discrimination and whether people think this wage gap is fair.

Read on >>

17 July 2021

We have no idea — same analysis, different results

In a recent paper, Akira Igarashi and James Laurence look at anti-immigrant attitudes in the UK and Japan. Like my 2019 paper in JEMS, they and how they highlight the limited research on non-Western countries, but they analysis they do is much more similar to what Sjoerdje van Heerden and I did in Urban Studies. Them like us relied on panel data to get a better handle on changing attitudes to immigrants. Them like us looked at the share of foreigners in the area (this relates to theoretical expectations that individual attitudes to immigrants reflect changes in the share of foreigners in the area; we refer to the same theories). We both used fixed-effect panel models. They find that “increasing immigration harms attitudes towards immigrant”, while we report that “a larger change in the proportion of immigrant residents is associated with more positive views on immigrants among natives” — yes, the exact opposite!

Need another example? Several studies examine the impact of sudden exposure to refugees on attitudes to immigrants and votes for radical-right parties. Such sudden exposure happened for example in Austria and Germany in 2015. In separate analyses, Andreas Steinmayr 2020 finds a clear increase in support for the radical-right, as we find in the work by Lukas Rudolph and Markus Wagner. Max Schaub, Johanna Gereke and Delia Baldassarri, by contrast “record null effects for all outcomes”. Same situation, same strategy to obtain the results.

We could now start the detective work, examining the small differences in modelling, ponder about the impact of how we define neighbourhoods, invoke possible differences between the countries (are the Netherlands an expectation, when the UK and Japan yield the same results? — not likely). Or we could admit how little we know, how much uncertainty there is in what we do, how vague our theories are in the social sciences that we can come to quite different conclusions in quite similar papers. I guess what we can see here is simply a scientific search for answers (it’s not like our research output would otherwise disagree so clearly). It’s probably also a call for more meta-level research: systematic analyses that synthesize what we do and don’t know, because even though individual papers sometimes contradict, we know quite a lot!

Heerden, Sjoerdje van, and Didier Ruedin. 2019. ‘How Attitudes towards Immigrants Are Shaped by Residential Context: The Role of Neighbourhood Dynamics, Immigrant Visibility, and Areal Attachment’. Urban Studies 56 (2): 317–34. https://doi.org/10.1177/0042098017732692.

Igarashi, Akira, and James Laurence. 2021. ‘How Does Immigration Affect Anti-Immigrant Sentiment, and Who Is Affected Most? A Longitudinal Analysis of the UK and Japan Cases’. Comparative Migration Studies 9 (1): 24. https://doi.org/10.1186/s40878-021-00231-7.

Rudolph, Lukas, and Markus Wagner. 2021. ‘Europe’s Migration Crisis: Local Contact and Out‐group Hostility’. European Journal of Political Research, May, 1475-6765.12455. https://doi.org/10.1111/1475-6765.12455.

Ruedin, Didier. 2019. ‘Attitudes to Immigrants in South Africa: Personality and Vulnerability’. Journal of Ethnic and Migration Studies 45 (7): 1108–26. https://doi.org/10.1080/1369183X.2018.1428086.

Schaub, Max, Johanna Gereke, and Delia Baldassarri. 2020. ‘Strangers in Hostile Lands: Exposure to Refugees and Right-Wing Support in Germany’s Eastern Regions’. Comparative Political Studies, September, 001041402095767. https://doi.org/10.1177/0010414020957675.

Steinmayr, Andreas. 2020. ‘Contact versus Exposure: Refugee Presence and Voting for the Far-Right’. The Review of Economics and Statistics, May, 1–47. https://doi.org/10.1162/rest_a_00922.

Zschirnt, Eva, and Didier Ruedin. 2016. ‘Ethnic Discrimination in Hiring Decisions: A Meta-Analysis of Correspondence Tests 1990–2015’. Journal of Ethnic and Migration Studies 42 (7): 1115–34. https://doi.org/10.1080/1369183X.2015.1133279.

17 August 2019

How well do correspondence tests measure discrimination?

Correspondence tests are a useful field experiement to measure discrimination in the formal labour market. These correspondence tests are also known as CV experiements: Researchers send two equivalent applications to an employer, differening only in the quantity of interest — gender and ethnicity are common. If only the majority or male candidate is invited for a job interview, we probably have a case of discrimination. Once we aggreate across many employers, we’re pretty confident to have captured discrimination.

Most studies stop there, declining any offer to reduce the burden on employers. The hiring process, however, does not end there. Lincoln Quillian and his team have now compiled a list of studies that went further. They find that the first stage of screening is far from the end of discrimination, and the job interview can increase overall discrimination substantially. Correspondece tests focusing on the first stage will capture only some of the discrimination. Interestingly the discrimination at the job interview stage appears unrelated to discrimination at the first screening of applications.

Quillian, L., Lee, J., & Oliver, M. (2018). Meta-Analysis of Field Experiments Shows Significantly More Racial Discrimination in Job Offers than in Callbacks. Northwestern Workin Paper Series, 18(28). Retrieved from https://www.ipr.northwestern.edu/publications/papers/2018/wp-18-28.html

Zschirnt, E., & Ruedin, D. (2016). Ethnic discrimination in hiring decisions: A meta-analysis of correspondence tests 1990–2015. Journal of Ethnic and Migration Studies, 42(7), 1115–1134. https://doi.org/10.1080/1369183X.2015.1133279

Image: CC-by Richard Eriksson.

19 January 2019

Ethnic discrimination in hiring: UK edition

The BBC report on a large correspondent test in the UK carried out by the excellent GEMM project. It’s good to see this reach a wider audience; it’s sad to see the results from our meta-analysis confirmed once again.

British citizens from ethnic minority backgrounds have to send, on average, 60% more job applications to get a positive response from employers compared to their white counterparts

What I really like about this short report by the BBC is that the essentials are covered. Yes we see discrimination, but no, it’s not so bad that none of the minority applicants would ever succeed. They also start the piece with an example of someone changing their name on the CV as a strategy to counter expected (or experienced) discrimination — and they highlight that discrimination has not declined despite policy changes, and indeed that discrimination affects native citizens who happen to have a ‘foreign’ name: they pay for an action of their parents or grandparents.

Are employers in Britain discriminating against ethnic minorities?, GEMM project: PDF of report

Zschirnt, Eva, and Didier Ruedin. 2016. ‘Ethnic Discrimination in Hiring Decisions: A Meta-Analysis of Correspondence Tests 1990–2015’. Journal of Ethnic and Migration Studies 42 (7): 1115–34. https://doi.org/10.1080/1369183X.2015.1133279.

16 September 2017

Discrimination not declining

A new meta-analysis draws on correspondence tests in the US to show that levels of ethnic discrimination in hiring do not seem to have changed much since 1989. This persistence in racial discrimination is bad news, and indeed Eva Zschirnt and I have shown the same result across OECD countries a year ago. While policies have changed, especially in the European Union, looking at the ‘average’ from correspondence tests suggests that they may not have been effective — and that is bad news.

Correspondence tests are widely accepted as a means to identify the existence of ethnic discrimination in the labour market, and as field experiments they are in a relatively good position to make the causal claims we typically want to make. It turns out that most correspondence tests have not paid sufficient attention to heterogeneity, which — as David Neumark and Judith Rich demonstrate — means that they likely over-estimate the degree of discrimination. Unfortunately, most old studies did not vary the groups in a way that this could be fixed post-hoc. If we throw these out of the meta-analysis, we probably no longer have sufficient studies to make claims about changes over time.

Meta-analyses are no doubt an important tool of science, but there’s always a delicate balance to be struck: are the experiments included really comparable? Here we’re looking at field experiments in different countries, different labour markets, different jobs, and different ethnic groups. We can control for these factors in the meta-analysis, but with the limited number of studies we have, this might not be sufficient to silence critics. With correspondence tests, we only cover entry-level jobs, and despite much more fine-graded studies going into the field recently, we don’t have a tool to really identify why discrimination takes place.

Neumark, David, and Judith Rich. 2016. ‘Do Field Experiments on Labor and Housing Markets Overstate Discrimination? Re-Examination of the Evidence’. NBER Working Papers w22278 (May). http://www.nber.org/papers/w22278.pdf.

Quillian, Lincoln, Devah Pager, Ole Hexel, and Arnfinn H. Midtbøen. 2017. ‘Meta-Analysis of Field Experiments Shows No Change in Racial Discrimination in Hiring over Time’. Proceedings of the National Academy of Sciences, September, 201706255. doi:10.1073/pnas.1706255114.

Image: CC-by CharlotWest