Challenges in International survey research: illustrations and solutions

Reviews my work on international mail surveys and response rates across countries

Anne-Wil Harzing - Mon 3 Apr 2017 17:08 (updated Wed 20 Dec 2023 16:57)

When doing my PhD in the mid nineties, I was struck with how little information was available on how to conduct international mail surveys. This was all the more surprising, since there are an overwhelming number of publications on the response rate effects of virtually every imaginable aspect of mail surveys in a domestic setting.

I found articles dealing with the number of questions, questionnaire length, the colour of the questionnaire, user friendly questionnaire formats, ticking versus circling answers, the name of the researcher, anonymity, deadlines, type of outgoing postage, type of return envelope, pre-contacts, follow-ups, offer of results, personalization, topic interest, auspices of the survey, numerous types of incentives, colour of the signature on a covering letter, use of handwritten postscripts and many, many more.

However, none of these dealt with international mail surveys. So wrote up my own experience of doing an international mail survey in subsidiaries in 22 countries during my PhD. Submitting the first version of this article (which was only my second academic article) led to one of my more disturbing experiences with the journal peer review process. One of the reviewers basically said: an academic journal is not a confessional, we are not interested in your problems, you can’t write, and you’ll never make it as an academic. I thus learned the first rule of academia: grow a thick skin and persist!

Response rates in international mail surveys

So I rewrote the paper to provide more focus on a single issue (response rates) and it was subsequently accepted and published in International Business Review. It has acquired a steady stream of citations ever since, and with nearly 500 citations is one of the 40 most cited articles published in the journal. It seems some people were interested in my problems after all :-).

The paper itself includes a fairly extensive description of procedures that might help to increase response rates. I still like my choice of incentive, the result of long discussions with my husband and a sudden flash of insight: a bag of Pickwick tea for one.

This tea-bag was attached to the cover letter next to a PS: “Why don’t you take a short break, have a nice cup of tea and fill out the questionnaire right now, it will only take 10-15 minutes”.

This incentive was hypothesised to catch the addressee’s attention, prevent the questionnaire from being thrown away immediately, bring the respondent in a pleasant mood and emphasise that it would not take too much time to fill out the questionnaire (just the time to drink a cup of tea). In the reminder, we elaborated on this theme by including instant coffee for the addressees that did not like tea.

What explains differences in response rates across countries?

In a subsequent publication I focused more specifically on differences in response rates between countries. My results show that, when compared to non-respondents, respondents were geographically and culturally closer to the Netherlands (my home country and the country from which the questionnaire was mailed) and were more internationally oriented. They also worked in smaller subsidiaries and in companies not listed on the Global Fortune 500 and came from countries with a lower level of power distance. In addition, there was some indication that English language capacity might be a factor influencing response rates as well.

Based on these results, the article made a wide range of recommendations for improving response rates in cross-national mail surveys. Most of these involve tailoring incentives and questionnaire design to different countries and cultures. The tea bag incentive for example might not have appealed equally to all nationalities. A colleague from Hong Kong suggested that Chinese addressees would perhaps not have returned the questionnaire, because they did not want the researcher to think they simply responded for the tea. She recommended not including any incentives at all for Chinese respondents.

Another colleague suggested that the tea bag incentive would never have worked in Saudi Arabia, since respondents would take it seriously and consider it too meagre a compensation for filling out the questionnaire, instead of taking it as the little joke for which is was intended. An example of two opposite, but equally negative responses to the same incentive!

puzzle

Putting it all together

In 2013, Sebastian Reiche, Markus Pudelko and I pulled together our experience in international survey research and provided a comprehensive and practical review in a paper published in the European Journal of International Management. We discussed every stage of the research process, including defining the study population, gaining data access, survey development, data collection, data analysis, and finally publication of the results.

For each stage, we review the pertinent literature, provide illustrations based on examples from our own research projects, and offer possible solutions to address the inherent challenges by formulating suggestions for improving the quality of international survey research. As EJIM’s proactive editor – Vlad Vaiman – realised the importance of making this paper widely accessible, it is available for free at the EJIM website. In 2023, it also became the winner of the European Journal of International Management Inaugural best paper award.

References

Harzing, A.W. (1997) Response rates in international mail surveys: Results of a 22 country study, International Business Review, 6(6): 641-665. Available online... - Publisher’s version
Harzing, A.W. (2000) Cross-national industrial mail surveys: Why do response rates differ between countries, Industrial Marketing Management, 29(3): 243-254. Available online... - Publisher’s version
Harzing, A.W.; Reiche B.S.; Pudelko, M. (2013) Challenges in international survey research: A review with illustrations and suggested solutions for best practice, European Journal of International Management, 7(1): 112-134. Available online... - Publisher's version (free access!). Winner of the European Journal of International Management Inaugural best paper award in 2023.

Journal of International Business Studies articles on methodological issues

[From the AIB SIG Research Methods website: https://rmsig.aib.world/resources-2/jibs-methods-editorials/]

Drop me a line

Free pre-publication versions of these papers are hyperlinked. If you’d like to have an official reprint for these papers, just drop me an email.

Anne-Wil Harzing is Emerita Professor of International Management at Middlesex University, London and visiting professor of International Management at Tilburg University. She is a Fellow of the Academy of International Business, a select group of distinguished AIB members who are recognized for their outstanding contributions to the scholarly development of the field of international business. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.

Anne-Wil Harzing's profile and contact details >>