Here’s a nice overview of causal inference by Scott Cunningham. Yes, you get an entire book as a free download, and it’s got you covered from probability to Pearl’s directed acyclical graphs, from instrumental variables to synthetic control. It comes across quite friendly, but has enough econometrics to scare many off. I quite enjoyed the historical bits thrown in here and there to explain where the methods came from.
Correlations are some of the basics in quantitative analysis, and they are well suited for graphical examination. Using plots we can see whether it is justified to assume a linear relationship between the variables, for example. Scatter plots are our friends here, and with two variables it is as simple as calling plot() in R:
If we have more than two variables, it can be useful to plot a scatter plot matrix: multiple scatter plots in one go. The pairs() command is built in, but in my view not the most useful one out there. Here we use cbind() to combine a few variables, and specify that we don’t want to see the same scatter plots (rotated) in the upper panel.
pairs(cbind(var1, var2, var3, var4) , upper.panel=NULL)
A more flexible method is provided in library(car) with the scatterplotMatrix(). If this is not flexible enough, we can always split the plot and draw whatever we need, but that’s not for today.
library(car)scatterplotMatrix(cbind(var1, var2, var3, var4))
If we have many more variables, it’s necessary to draw multiple plots to be able to see what is going on. However, sometimes after having checked that the associations are more or less linear, we’re simply interested in the strength and direction of the correlations for many combinations of variables. I guess the classic approach is staring at a large table of correlation coefficients, but as is often the case, graphics can make your life easier, in this case library(corrplot):
This is certainly more pleasant than staring at a table…
For all these commands, R offers plenty of ways to tweak the output.
There’s a new book edited by S. Michael Gaddis on audit studies. The subtitle promises to go behind the scenes with theory, method, and nuance — and this is what the book provides. As such, the book is a much needed contribution to the literature, where we typically see the results and little how we got there. With (not so) recent concerns around researcher degrees of freedom, the tour behind the scenes offered by the various chapters are an excellent way to make visible and apparent the ‘undisclosed flexibility’ as Simmons et al. called it in 2011. It’s one thing to discuss this in abstract terms, and it’s another thing to sit down with actual research and reflect on the many choices we have as researchers. Indeed, public reflection on research practices may be relatively rare in itself when it comes to quantitative research.
The book comes with a dedicated support webpage: http://auditstudies.com/ (do me the favour to update the “coming soon” banner). On this website, several chapters can be downloaded as pre-prints, though it’s not all the contents if someone is looking for a free book. I hope the authors will make their code available on the website as promised in several places in the book, because this will be another greatly helpful resource for those new to audit studies or looking for new directions.
I greatly enjoyed to read the reflections by other researchers doing audit studies, and would definitely recommend the book to anyone thinking of doing an audit study. At times there were passages that seemed a bit redundant to me, but all the chapters are written in such an accessible way that this didn’t bother me much. Where I think the book falls a bit short is on two fronts. First, it is very US-centric. In itself this is not an issue, but there are several instances where the authors don’t reflect that perhaps in other countries the markets are not organized the same way. In my view, a comparison to other countries and continents would have been fruitful to underline some of these assumptions — I’ve tried to just this on attitudes to immigrants. Second, the book is not a guidebook. I know, it doesn’t claim to be one, but the book asks so many (justified) questions and offers comparatively few concrete guidelines like Vuolo et al. offer it on statistical power. In this sense, the book will stimulate readers to think about their own research design and not provide a template. And this is actually a good thing, because as the chapters make apparent without normally saying so, there is no universal approach that suits different markets in different places and at different times.
So, should you buy the book? Yes if you want to carry out your own audit study, yes if you want to better understand and qualify the results of audit studies, and yes if you’re looking for guidelines — because the book will make you realize that you’re largely on your own. What would probably useful, though, would be a checklist of things to consider, something readers will have to create themselves on the basis of chapters 4 (Joanna Lahey and Ryan Beasley), 5 (Charles Crabtree), and 6 (Mike Vuolo, Christopher Uggen, and Sarah Lageson).
Gaddis, S. Michael, ed. 2018. Audit Studies: Behind the Scenes with Theory, Method, and Nuance. Methodos 14. New York: Springer. https://www.springer.com/cn/book/9783319711522
Ruedin, Didier. 2018. ‘Attitudes to Immigrants in South Africa: Personality and Vulnerability’. Journal of Ethnic and Migration Studies. https://doi.org/10.1080/1369183X.2018.1428086.
Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2011. ‘False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant’. Psychological Science 22 (11): 1359–66. https://doi.org/10.1177/0956797611417632.
Vuolo, Mike, Christopher Uggen, and Sarah Lageson. 2016. ‘Statistical Power in Experimental Audit Studies: Cautions and Calculations for Matched Tests With Nominal Outcomes’. Sociological Methods & Research, 1–44. https://doi.org/10.1177/0049124115570066.
Zschirnt, Eva, and Didier Ruedin. 2016. ‘Ethnic Discrimination in Hiring Decisions: A Meta-Analysis of Correspondence Tests 1990–2015’. Journal of Ethnic and Migration Studies 42 (7): 1115–34. https://doi.org/10.1080/1369183X.2015.1133279.>/small>
We’re now hiring a postdoctoral researcher (4 years, 70% FTE) for a project on overcoming inequalities and ethnic discrimination in the labour market. The project is jointly with Wassilis Kassis. You’ll be working at the University of Neuchâtel, and will be joined by a doctoral students by the end of the year. Full advert here: http://nccr-onthemove.ch/wp_live14/wp-content/uploads/2018/03/IP26-Jobs-NCCR-Phase-II-UNINE-PD.pdf
I’ve long been critical of population estimates as ‘evidence’ of racism, but now there is no reason left to do so. The basic ‘evidence’ is as follows: There are say 5% immigrants in country X, you ask the general population, and their mean estimate is maybe that there are 15% immigrants in the country. Shocking, they overestimate the immigrant population, which is ‘evidence’ that the general population is generally racist (I enjoyed this phrase). I’ve been critical of this because of three reasons. First, we don’t generally tell survey participants what we mean by ‘immigrants’, but use a specific definition (foreign citizens, foreign born) for the supposedly correct answer. Second, why should members of the general population have a good grasp of the size of the immigrant population? We might be able to estimate the share of immigrants in our personal network, but that’s not the same as estimating population shared. Third, if we see this as evidence of racism, we assume that the threat perspective is dominant.
It turns out, however, that there is a general human tendency to overestimate the population share of small groups: immigrants, homosexuals, you name it. David Landy and colleagues demonstrate that this tendency to overestimate small groups comes hand in hand with a tendency to underestimate large groups — a pull towards the average. There’s nothing particular about immigrants there, and nothing about racism either.
Landy, D., B. Guay, and T. Marghetis. 2017. ‘Bias and Ignorance in Demographic Perception’. Psychonomic Bulletin & Review, August, 1–13. https://doi.org/10.3758/s13423-017-1360-2.
Photo: CC-by-nc-nd by IceBone