Our new study examines how residents in Switzerland perceive migration-related social change in their municipality, their place of work, and in public. We left the ivory tower and listened. The result is a detailed and diverse picture: Migration is perceived as part of social change more widely, but it’s not migration as such that evokes threat. Perceptions of threat and fear are a side-effect of wider social change and economic growth, such as changes to the built environment because of new buildings, cars and transportation, and a perceived impoverishment of social life. It is clear that a majority seek communities with local opportunities to meet and exchange, but many also recognize that the world changes.
Isolation measures to contain the spread of COVID-19 means that social researchers who conduct face-to-face fieldwork (interviews, focus groups, participant observation, ethnographies etc) are now faced with the challenge of either delaying or re-inventing their methods so that they can continue their research until these measures are relaxed.
I have recently explored open-source approaches to computer-assisted qualitative data analysis (CAQDA). As is common with open-source software, there are several options available, but as is often also the case, not many of them can keep up with the commercial packages, or are abandoned.
Here I wanted to highlight just three options.
RQDA is built on top of R, which is perhaps not the most obvious choice — but can have advantages. The documentation is steadily improving, making it more apparent how RQDA has the main features we’ve come to expect from CAQDA software. I find it a bit fiddly with the many windows that tend to be opened, especially when working on a small screen.
Colloquium is Java-based, which makes it run almost everywhere. It offers a rather basic feature set, and tags can only be assigned to lines (which also implies that lines are the unit of analysis). Where it shines, though, is how it enables working in two languages in parallel.
CATMA is web-based, but runs without flash — so it should run pretty anywhere. It offers basic manual and automatic coding, but there’s one feature we really should care about: CATMA does TEI. This means that CATMA offers a standardized XML export that should be usable in the future, and facilitate sharing the documents as well as the accompanying coding. That’s quite exciting.
What I find difficult to judge at the moment, is whether TEI will be adopted by CAQDA software. Atlas.ti does some XML, but as far as I know it’s not TEI. And, would TEI be more useful to future researchers than a SQLite database like RQDA produces them?
Qualitative studies are often described as small N studies because the number of respondents is small. I argue that this is the wrong perspective: What we really have in qualitative data, say interviews, is lots of data (points) clustered within individuals. Rather than focusing on the number of respondents, we should probably focus on the number of relevant statements (i.e. statements about our quantity of interest), and describe this number (along with the number of respondents). When computer aided qualitative data analysis (CAQDA) is used, I guess the number of tags relevant to our quantity of interest is that number. Seen this way, many qualitative studies are no longer small N studies, but we’re still faced with unstructured, messy data that may be difficult to analyse, and of course we don’t have independent observations — so generalization remains a challenge.
As an interdisciplinary institute, we have recently discussed whether (and under what circumstances) we should pay interview respondents and participants in our studies? Here are a few things I have compiled for this purpose. At this point, I really would like to thank the participants of the Rencontre Scientifique SFM on 28 June 2016 for the discussion and comments.
General Ethical Principles
General ethical principles apply, and the following table reviews a list of general ethical principles taken from the SFM Ethics Guidelines in view of the question of paying interview respondents. The focus is on interview participants in a wider sense to include focus groups, expert interviews, laypersons with expert knowledge, and informants.
Impact of Payment
no harm to subjects and researcher
not affected by payment
potential benefits to subjects
payment clearly as a benefit to subjects
informed consent should normally be obtained: participants should be aware of nature of research and their involvement; participants have a right to withdraw consent at any time without giving any reason
payment creates incentives to participate when there is no consent; so payment needs to be separated from completion
accordance to relevant law and legislation
payment is legal, but may be taxable
researchers must respect the rights, dignity, and interests of participants, including assurances of confidentiality and anonymity
payment does not affect these obligations on part of the researcher
research involving children should obtain consent from both the parents and the children, consistent with their capacity
payment does not affect this
reduce likelihood that research experience is disturbing to participants and others
payment does not change this, but may compensate for inadvertent violation of this principle
avoid actions that may have deleterious consequences for researchers who come after or undermine reputation of the discipline
payment may increase the expectation that other researchers pay (which can undermine future researchers to carry out the same kind of research), payment as such is unlikely to undermine the reputation of the discipline, but may enhance it
ensure that funders appreciate the ethical obligations of researchers
not affected, if there is an ethical obligation to pay, this needs to be communicated to the funder – the fact that the project will be more expensive than a competitor’s is no excuse not to follow research ethics
Disciplinary Traditions in Experiments
In experiments, economists almost always pay their participants. Economists worry that without payment there is no real incentive to follow the instructions. With payment there is a possible problem with satisficing.
In psychology, they hardly ever do. Psychologists worry that with payment inherent preferences and motivations are overruled. Without payment there is a potential problem with participants trying to please the researchers.
Participants and their contributions should be respected.
Impact of Payment
Respondents in professional capacity are already paid
payment not necessary
Respondents in professional capacity may be ordered to participate
People like to talk (about themselves)
payment not necessary
No obligation to participate, but payments can be interpreted as coercion1 (Boddy et al. 2010), people from poorer background may be more susceptible to this kind of influence
undermines informed consent, especially for some parts of society
Payment can reduce non-response bias, precisely because payment incites participation (Boddy et al. 2010; Grady 2011).
Paying creates an obligation that can blur boundaries and undermine trust.
May skew samples (Grady 2011)
Not paying may introduce bias by excluding participants, e.g. poor who cannot afford to participate, despite having something important to say (Thompson 1996)
reduce coverage bias
Boddy et al. (2010) suggest: (1) create guidelines when and how payments are made, (2) payment is justified, (3) ensure that those who withdraw are still paid, (4) carefully consider cases where consent may be given only because of the payment
In the social sciences it is uncommon to pay interview participants. This, however, does not constitute an ethic statement. Direct costs incurred by participants (e.g. travel expenses) should always be reimbursed.
Sometimes it is necessary to pay to ensure participation (e.g. interviews with prostitutes or taxi drivers (e.g. Gambetta and Hamill 2005), some online survey panels). A similar case is paying for survey respondents, where payment may be necessary to get access.
There are some ideal-typical cases of participants, but in reality the boundaries are fluid: (1) Focus groups, (2) experts in their professional capacity, (3) lay persons with expert knowledge, and (4) informants.
(1) Focus Groups
Participants in focus groups are normally paid. Participants are generally paid a flat amount (depending on the circumstances) to cover travel and a symbolic amount. Focus group participants need to prepare for the focus group, they are asked to come to the venue the researchers specify at a given time (in the other cases considered here, it is the researcher who travels and adjust his or her schedule). The participants are asked to follow the design and have less scope to deviate from it. Payment also creates an informal obligation to turn up, and often motivates participants who would otherwise not participate in such an endeavour despite supporting the research project otherwise: the specificities of a focus group (time, scheduling, and preparations) make it relatively costly to the participants.
(2) Experts in their Professional Capacity
Experts in their professional capacity are not normally paid. There is no reason to. Experts in their professional capacity are in a way paid to participate, but potential direct expenditures (e.g. travel) should be reimbursed if their employer does not cover them. Experts are free to tell the researchers what they want and have more scope to deviate from the questionnaire and participants in a focus group.
(3) Lay Persons with Expert Knowledge
Lay persons with expert knowledge are not normally paid. Lay persons with expert knowledge are like experts in that they are consulted as witnesses and for providing a synthesis – describing the situation of a group, for instance, not (just) about the person in question. They may be active in associations, often on a voluntary basis. At the same time, given their position and expert knowledge, they are often frequently consulted by researchers. They are free, however, to decline participation in a way experts in their professional capacity may not be. They often agree to participate because they have a message to share, and their contribution should be sufficiently recognized. Like with other participants, direct costs (e.g. travel) should always be reimbursed. Lay persons with expert knowledge are free to tell the researchers what they want and have more scope to deviate from the questionnaire and participants in a focus group.
Informants are not normally paid. Informants are free to decline participation. However, informants can be in a precarious situation (e.g. undocumented migrants). Researchers frequently provide symbolic gestures like vouchers in these situations, but in order not to create pressure to participate, these gestures should probably not be mentioned during recruitment: informants should not participate only or largely because of the rewards. Informants are free to tell the researchers what they want and have more scope to deviate from the questionnaire than participants in a focus group, and contrary to experts and lay persons with expert knowledge, no synthesis is expected on part of the informant.
Should sensitive topics mean more payment? – There is no reason why this should be the case, if anything larger payments can lead to distorted answers.
Should those with higher income receive more payment? – No, payments are allowances and not payments to cover (potential) loss of earnings. The situation is different where access to a group of informants is contingent on payment (e.g. prostitutes).
The decision whether to pay has to be determined on a case-by-case basis
We now recommend that researchers consult the specific consideration above when considering whether to pay participants or not.
1 payment is not coercion (no harm threatened), but may be undue inducement/influence (an offer one cannot refuse, controlling or irresistible influence, strong enough to compel participation against interest) (Grady 2011)
2 money may affect risk assessments and consent (Grady 2011)
3 Free prize draws are frequently used in marketing (not subject to employment laws, not subject to lottery laws). The nature of prizes, cash equivalents, and notification of winners must be clear (Boddy et al. 2010).
4 Film-makers do not pay participants, it’s regarded a “privilege” to tell one’s story, although travel expenses are generally reimbursed; usually there is no money in documentaries; they worry that if you pay, people will tell you what they think you want to hear; experts should be credited; most experts are happy to contribute; when filming with really poor populations, the crew often make a gesture after filming, e.g. food, clothes. Film-makers are worried that paying in advance (or agreeing to pay) leads to fictional accounts.
5 Journalists typically don’t pay. They worry that it affects what people say.
Boddy, J., T. Neumann, S. Jennings, V. Morrow, P. Alderson, R. Rees, and W. Gibson. 2010. The Research Ethics Guidebook: A Resource for Social Scientists. http://www.ethicsguidebook.ac.uk/.
Gambetta, D., and H. Hamill. 2005. Streetwise: How Taxi Drivers Establish Their Customers’ Trustworthiness. New York: Russel Sage Foundation.