Lessons for Conducting Phone-based Surveys During Covid-19 and Beyond

The lived experiences of the Covid-19 pandemic are starkly divided along gender lines, and the pandemic has already exacerbated gender inequalities within the home and in the labour market across countries. To understand the gendered impacts and experiences of the pandemic in urban Pakistan and inform gender-sensitive policy responses, researchers from the Institute of Development Studies and IDEAS (working under the auspices of the Action for Empowerment and Accountabilityprogramme) conducted a phone-based survey with men and women from nearly 1500 households in Lahore during September and October 2020. This blog documents lessons learned from the survey process, highlighting some challenges and adaptations specific to surveying women, which we hope will be useful to other researchers.

Past in-person contact helped build lasting trust

Establishing trust with respondents is necessary for recruiting them to participate research, and also key for ensuring the quality of survey responses. At the outset, we were concerned that such trust might be harder to build over the phone than in an in-person interaction, leading to lower response rates.

However, we were able to achieve a high response rate of nearly 80 per cent in our survey. This was in part because our sampling strategy involved recontacting respondents who had previously participated in two rounds of in-person surveys in 2018. Enumerators reported that they found it relatively easy to establish trust with the respondents, many of whom recalled being surveyed two years ago.

Importantly, our sample was restricted to households where at least one respondent from the household had agreed to provide a contact number when previously surveyed.This meant that respondents who were truly unwilling to be recontacted and chose not to provide a phone number (about 5 per cent of the original sample) were already screened out.

Providing token compensation (Rs.100) for participation in the past survey rounds also contributed to establishing credibility with respondents. Enumerators reported that some respondents even remembered names of the enumerators who had personally delivered the compensation to them in 2018.

One enumerator per household is better than two

In-person surveys conducted by IDEAS in the past have largely relied on female enumerators to survey women, and male enumerators to survey men. This is to avoid violating norms of social interaction between women and non-family member men, which may negatively affect survey participation, and even put female respondents at risk of backlash from male household members.

However, this poses a challenge in phone surveys in a context where women are far less likely to personally own cell phones. At the pilot stage, female enumerators reported that mostly men answered the phone, and speaking to women necessitated at least a short conversation with a man.

For past in-person interviews, female enumerators have often visited households during the daytime, when men are likely to be at work and children are at school, so as to have private, uninterrupted time with women respondents. This is not possible when women have to be reached through phones that are owned/controlled by men. Thus, a phone-based strategy necessarily implies some degree of mediated, rather than direct access to women.

At the pilot stage, male enumerators interviewing male respondents reported that they were losing the opportunity to speak with female respondents who were available for an interview, but were unwilling to speak to a male enumerator. Male and female enumerators, who could not be in the same physical space due to social distancing measures, found it hard to coordinate amongst themselves about the order and timing of calls to the same household.

We thus used the same (female) enumerator to interview both the male and female respondent in each household. While it may have been inappropriate for male enumerators to ask to speak privately with female respondents, male respondents in our sample were comfortable speaking with female enumerators. In fact according to one of the female enumerators, ‘men gave more elaborate answers than women, who were sometimes shy’, another reported ‘female [respondents] often gave shorter and to the point responses compared to their male counterparts’.

Privacy is not gender neutral

Phone-based interviews can provide an opportunity for greater respondent privacy, since household members who may be present in the vicinity cannot hear the enumerator’s questions.

However, this privacy may be violated of course if the call is on speakerphone. In a phone based survey with Self-Employed Women’s Association (SEWA) members in Gujrat, India conducted in May-June, 65 per cent of surveyed women had the phone on speaker for at least a portion of the interview. In our study, survey enumerators recorded their perception of whether the call was on speakerphone for all or some portion of each interview. We document a gender gap in this measure of privacy. While we find that while the proportion of calls on speakerphone was overall much lower than the Gurjrat survey, it was twice as high for surveys conducted with women (18 per cent) compared to those conducted with men (8 per cent)  within our sample.

Ensuring ongoing consent requires active measures

The procedure for seeking oral consent for participation a survey is largely common to in-person and phone surveys. Enumerators are trained to emphasise the ongoing aspects of consent: that respondents can choose to opt out at any point, and that they can refuse to answer any individual questions.

However, when conducting in-person interviews, enumerators also rely on a number of cues, and are trained to adapt on the spot to ensure continued respondent comfort. For instance, upon noticing other household members entering the space or being in hearing range, an enumerator may pause the interview momentarily or change the subject, especially if the topic being discussed is sensitive. This is a common practice when surveying women about issues of domestic violence.

While we did not ask questions about domestic violence in our survey, we did ask women about their autonomy within the home, access to pre/post natal care, and ways in which their lives have changed due to men’s increased presence in the home. These may be topics which women are uncomfortable discussing openly without privacy. Prior to asking these questions, enumerators used the following script:

“We would like to ask some specific questions about women’s lives and health at this time. You will be able to answer just yes or no mostly. Are you comfortable answering these questions, and do you have privacy at the moment?”

In response to this, 70 per cent of women in our sample reported they had privacy and were comfortable answering; 25 per cent of women said they were comfortable answering, even though they did not have full privacy at the time, and 5 per cent of women said they would be uncomfortable answering. These 5 per cent of women thus opted out of the survey module.

Given our earlier findings on gender gaps in access to privacy, coupled with enumerators’ inability to gauge respondent comfort through in-person cues, we relied on a strategy of actively seeking verbal consent during the survey before asking a relatively sensitive set of questions.

In upcoming blog posts, we will report on findings from the survey on respondents’ exposure to Covid-19, shocks to income and the burden of carework in sample households, gender gaps in health related attitudes and behaviors, and in access to sources of information and support during the pandemic.

This blog post was published on the IDS platform at the following link: https://www.ids.ac.uk/opinions/lessons-for-conducting-phone-based-surveys-during-covid-19-and-beyond/