Ethics versus Permissions

Today we’ve been discussing ethics and research. I’m very happy to see ethics being discussed in research articles, but from the perspective of someone not in an environment ‘governed’ by IRB decisions, we’re following the developments with some concern.

Let me be clear, ethics in research is a good and essential part of what we’re doing. What is worrying, though, is the formalization of ethics decisions to the extent that a commission decides and approves which research is ethically legitimate and gets a permission to go ahead. No permission, no research.

Increasingly, journals ask for IRB approval when we submit our research to them. To the extent that this encourages a discussion of research ethics and practices to match, I welcome this. To the extent that it takes one way of doing research ethics for granted (the way of IRB approvals), I’m not so sure.

A challenge in interdisciplinary panels is that we mean quite different things when we use the same terminology, like “covert research”. Because it’s formalized, there is a real risk that the instruments we use for ethical research — like informed consent forms — become a principle in themselves, not the underlying concerns for the respect for people. With that, we drive researchers to find creative ways to fulfil the formal requirements, but we do not necessarily encourage them to think about the ethical implications of the research.

When we’re in the logic of permissions and approvals, the incentives for the researchers are simply to follow a certain procedure. For the institutions, the incentives are to minimize the risk of being sued, and this may not necessarily align with ethical research practices. Will we soon have to submit a DOI for the approvals when we submit to journals as proof that we’ve followed the procedures, just so that we can demonstrate we’re not to blame? It won’t be about ethical guidance when we feel we need it, or a comforting second opinion, but a matter of form. Is there still time to take matters in our own hands and design research ethics from the bottom up? Or is the IRB way inevitable?

Salganik, Matthew J. 2017. Bit by Bit: Social Research in the Digital Age. Princeton: Princeton University Press.

Making an online lecture

Colin Mills has posted some useful advice on preparing online lectures using PowerPoint.

It’s a time consuming job to get even passable results. After I had written the script – which could easily take me 8-12 hours (and I already had the slides prepared) it would take 3 hours to record 50 minutes of material plus the time to turn the Powerpoints into a videos & upload them to YouTube.

The results are here.

National Study on Ethnic Discrimination in the Housing Market

Today I’m giving you the first national field experiment on ethnic discrimination in the housing market. Financed by the Swiss Office for Housing and the NCCR on the move, we examined to what extent one’s name affects the likelihood to be invited to view an apartment. We covered the entire country, across language regions and across urban and rural areas.

Between March and October 2018, our diligent research assistants sent more than 11,000 enquiries to over 5,700 landlords in all parts of Switzerland. We varied the name of the person sending an enquiry (stimulus sampling) along with other features such as politeness or the family situation. Overall over 70% of the enquiries were answered positively in the sense of an invitation to view the apartment or steps in this direction.

We find no clear differences between commercial and private landlords. The response rate for women was around 1 percent higher, while highly qualified people had a 2 percent higher response rate, especially academics who use their doctoral title (we dind’t expect this to make such a big difference when we designed the study). As previous field experiments have shown, the quality of the message we sent affected the probability of a response: Compared to a standard text, the response rate for friendlier queries is about 5 percent higher, while queries with the default text from online portals show a 10 percent lower response rate.

We find evidence of ethnic discrimination in the sense of unequal treatment based on the name. Enquiries with names from neighbouring countries (Germany, Italy, France) were even invited somewhat more frequently to view apartments than those from Switzerland, but people with Kosovar (response rate just under 3 percent lower) or Turkish names (response rate about 5 percent lower) have significantly fewer chances of being invited for a viewing. Whether those interested were naturalised with foreign-sounding names or stated that they had a permanent residence permit was hardly a factor. The rate of discrimination we observe is similar in order of magnitude to that found in comparable studies in other Western countries.

With the national coverage, we can also observe variation in responses by local context where the property is located. In municipalities with higher rental prices, the positive response rate is higher for everyone, and a higher vacancy rate in the municipality is associated with a higher response rate, except for people with Kosovar names. In urban areas the probability of discriminating against people with foreign names is lower. We also find that people with foreign-sounding names are less likely to be invited in municipalities with restrictive political attitudes towards immigration (as measured in the results of popular initiatives and referendums).

Auer, Daniel, Julie Lacroix, Didier Ruedin, and Eva Zschirnt. 2019. ‘Ethnische Diskriminierung auf dem Schweizer Wohnungsmarkt’. Grenchen: BWO.

Audit Studies — The Book

There’s a new book edited by S. Michael Gaddis on audit studies. The subtitle promises to go behind the scenes with theory, method, and nuance — and this is what the book provides. As such, the book is a much needed contribution to the literature, where we typically see the results and little how we got there. With (not so) recent concerns around researcher degrees of freedom, the tour behind the scenes offered by the various chapters are an excellent way to make visible and apparent the ‘undisclosed flexibility’ as Simmons et al. called it in 2011. It’s one thing to discuss this in abstract terms, and it’s another thing to sit down with actual research and reflect on the many choices we have as researchers. Indeed, public reflection on research practices may be relatively rare in itself when it comes to quantitative research.

The book comes with a dedicated support webpage: (do me the favour to update the “coming soon” banner). On this website, several chapters can be downloaded as pre-prints, though it’s not all the contents if someone is looking for a free book. I hope the authors will make their code available on the website as promised in several places in the book, because this will be another greatly helpful resource for those new to audit studies or looking for new directions.

I greatly enjoyed to read the reflections by other researchers doing audit studies, and would definitely recommend the book to anyone thinking of doing an audit study. At times there were passages that seemed a bit redundant to me, but all the chapters are written in such an accessible way that this didn’t bother me much. Where I think the book falls a bit short is on two fronts. First, it is very US-centric. In itself this is not an issue, but there are several instances where the authors don’t reflect that perhaps in other countries the markets are not organized the same way. In my view, a comparison to other countries and continents would have been fruitful to underline some of these assumptions — I’ve tried to just this on attitudes to immigrants. Second, the book is not a guidebook. I know, it doesn’t claim to be one, but the book asks so many (justified) questions and offers comparatively few concrete guidelines like Vuolo et al. offer it on statistical power. In this sense, the book will stimulate readers to think about their own research design and not provide a template. And this is actually a good thing, because as the chapters make apparent without normally saying so, there is no universal approach that suits different markets in different places and at different times.

So, should you buy the book? Yes if you want to carry out your own audit study, yes if you want to better understand and qualify the results of audit studies, and yes if you’re looking for guidelines — because the book will make you realize that you’re largely on your own. What would probably useful, though, would be a checklist of things to consider, something readers will have to create themselves on the basis of chapters 4 (Joanna Lahey and Ryan Beasley), 5 (Charles Crabtree), and 6 (Mike Vuolo, Christopher Uggen, and Sarah Lageson).

Gaddis, S. Michael, ed. 2018. Audit Studies: Behind the Scenes with Theory, Method, and Nuance. Methodos 14. New York: Springer.

Ruedin, Didier. 2018. ‘Attitudes to Immigrants in South Africa: Personality and Vulnerability’. Journal of Ethnic and Migration Studies.

Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2011. ‘False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant’. Psychological Science 22 (11): 1359–66.

Vuolo, Mike, Christopher Uggen, and Sarah Lageson. 2016. ‘Statistical Power in Experimental Audit Studies: Cautions and Calculations for Matched Tests With Nominal Outcomes’. Sociological Methods & Research, 1–44.

Zschirnt, Eva, and Didier Ruedin. 2016. ‘Ethnic Discrimination in Hiring Decisions: A Meta-Analysis of Correspondence Tests 1990–2015’. Journal of Ethnic and Migration Studies 42 (7): 1115–34.>/small>