Everyone seems to be an epidemiologist these days. I have long lost count on the surveys that land in my inbox. It’s clear that the internet has made it very cheap to field surveys, especially surveys where questions of sampling don’t seem to be relevant to those fielding the surveys. It’s also clear that tools like SurveyMonkey and Qualtrics make it easy to field surveys quickly. But that’s no excuse for some of the surveys I’ve seen:
- surveys with no apparent flow between questions
- surveys where the e-mail makes it clear that they are desperate to get any answers
- surveys with incomplete flow logic (see example below)
- surveys that ask hardly anything about the respondent (like age, sex, education, location)
- surveys that throw in about any instrument that could be vaguely related to how people respond to Covid-19 (with no apparent focus; which is bound to find ‘interesting’ and statistically ‘significant’ results)
- double negatives in questions
- two questions in one
For example, how should I answer this required question at the bottom here? What if I assume corruption is evenly spread across all sectors, or not present at all?
I understand that we want to get numbers on the various ways Covid-19 affected us, but with surveys like these we’re not going to learn anything because they do not allow meaningful inferences. In that case, it’s sometimes better not to run a survey then pretending to have data.
I’m sure I’m not the first to notice, but it seems to me that peer-review encourages p-hacking. Try this: (1) pre-register your analysis of a regression analysis before doing the analysis and writing the paper (in your lab notes, or actually on OSF). (2) Do the analysis, and (3) submit. How often do we get recommendations or demands to change the model during the peer-reviewing process? How about controlling for X, should you not do Y, or you should do Z, etc.
Unless we’re looking at a pre-registered report, we’re being asked to change the model. Typically we don’t know whether these suggestions are based on theory or the empirical results. In the former case, we should probably do a new pre-registration and redo the analysis. Sometimes we catch important things like post-treatment bias… In the latter case, simply resist?
And as reviewers, we should probably be conscious of this (in addition to the additional work we’re asking authors to do, because we know that at this stage authors will typically do anything to get the paper accepted).
Photo credit: CC-by GotCredit – https://flic.kr/p/Sc7Dmi
Funny thing: Just as I’m doing some light reading on metrics, impact factor gaming, and predatory journals with no real peer-review, I get a revised article to review where there were six (!) reviewers solicitated, all of whom made substantial recommendations.
There is no shortage of books on academic writing. If you cannot decide where to start, in my view, you should start with “Write No Matter What” by Joli Jensen. Here’s why:
- it’s relatively short
- it summarizes the best advice out there
- it’s realistic
Like other books on academic writing, it starts by addressing common myths about academic writing. I find it painful to see these myths repeated in my own environment. In Jensen’s book, you’ll learn three taming techniques (creating a project box, using a ventilation file, and writing at least 15 minutes every day). So we’re looking at being organized, being realistic (i.e. having room for frustration, writing blocks, etc.), and that important continuing contact with the writing project.
Compared to other similar books I know, I really liked how “Write No Matter What” does not imply that if only you were more disciplined, you’d get all that writing done. No, instead there is an entire section on maintaining momentum, lost trails, and handling revisions and rejections. Getting stalled? There’s an entire chapter on that.
I didn’t enjoy the chapters on writing support that much, but if you’re looking into setting up a campus-wide (or even faculty-wide) writing support, you’ll get plenty of ideas what may or may not work.
Writing style is explicitly not covered, and I think that’s a good thing. Not that books on good writing were redundant — to the contrary! — but this way we get a focused book that can serve everyone from a first-year PhD student to established faculty.