Tag: research design

There’s a new book edited by S. Michael Gaddis on audit studies. The subtitle promises to go behind the scenes with theory, method, and nuance — and this is what the book provides. As such, the book is a much needed contribution to the literature, where we typically see the results and little how we got there. With (not so) recent concerns around researcher degrees of freedom, the tour behind the scenes offered by the various chapters are an excellent way to make visible and apparent the ‘undisclosed flexibility’ as Simmons et al. called it in 2011. It’s one thing to discuss this in abstract terms, and it’s another thing to sit down with actual research and reflect on the many choices we have as researchers. Indeed, public reflection on research practices may be relatively rare in itself when it comes to quantitative research.

The book comes with a dedicated support webpage: http://auditstudies.com/ (do me the favour to update the “coming soon” banner). On this website, several chapters can be downloaded as pre-prints, though it’s not all the contents if someone is looking for a free book. I hope the authors will make their code available on the website as promised in several places in the book, because this will be another greatly helpful resource for those new to audit studies or looking for new directions.

I greatly enjoyed to read the reflections by other researchers doing audit studies, and would definitely recommend the book to anyone thinking of doing an audit study. At times there were passages that seemed a bit redundant to me, but all the chapters are written in such an accessible way that this didn’t bother me much. Where I think the book falls a bit short is on two fronts. First, it is very US-centric. In itself this is not an issue, but there are several instances where the authors don’t reflect that perhaps in other countries the markets are not organized the same way. In my view, a comparison to other countries and continents would have been fruitful to underline some of these assumptions — I’ve tried to just this on attitudes to immigrants. Second, the book is not a guidebook. I know, it doesn’t claim to be one, but the book asks so many (justified) questions and offers comparatively few concrete guidelines like Vuolo et al. offer it on statistical power. In this sense, the book will stimulate readers to think about their own research design and not provide a template. And this is actually a good thing, because as the chapters make apparent without normally saying so, there is no universal approach that suits different markets in different places and at different times.

So, should you buy the book? Yes if you want to carry out your own audit study, yes if you want to better understand and qualify the results of audit studies, and yes if you’re looking for guidelines — because the book will make you realize that you’re largely on your own. What would probably useful, though, would be a checklist of things to consider, something readers will have to create themselves on the basis of chapters 4 (Joanna Lahey and Ryan Beasley), 5 (Charles Crabtree), and 6 (Mike Vuolo, Christopher Uggen, and Sarah Lageson).

Gaddis, S. Michael, ed. 2018. Audit Studies: Behind the Scenes with Theory, Method, and Nuance. Methodos 14. New York: Springer. https://www.springer.com/cn/book/9783319711522