Language effects in international mail surveys

Explains how questionnaire language might influence the responses received in cross-cultural research

Anne-Wil Harzing - Sun 10 Jun 2018 09:10 (updated Sun 24 Apr 2022 10:53)

Languages have always fascinated me, right from the days of my teens when I studied six languages at secondary school (Dutch, English, French, German, Latin and Greek). I graduated in five of them, adding only Geography and History to my selection and taking Economics as a supplementary. It was what was – and apparently still is – in rather derogatory terms called a “pretpakket”, i.e. the “easy” option, taken only by pupils who have no ability in Science subjects.

In my case, lack of ability in Science subjects was probably not the main reason for chosing languages; I was quite decent in Maths and did well in Chemistry too. But I didn’t really enjoy Maths, probably partly due to my teacher who didn’t think Maths was for girls; yes it was the seventies... Not graduating in Mathematics meant, however, that I had closed the door on most university degrees – including Economics & Business – which was one of the reasons I opted for a vocational degree. The other reason was that none of the older generation in my family and only two or three of my 40-odd cousins ever tried university. So university just didn’t feature on my horizon.

Hence, I did a vocational degree in Business & Languages for 3 ½ years, learning how to write business letters in French, English, and Spanish and getting the basics of Marketing, Finance & Accounting, and Management. Then, after being expertly coached in Mathematics by my boyfriend – an electrical engineer – I passed a “deficiency course” in Mathematics with a 10 out of 10 and went on to study Economics & Business for another four years after all. But language continued to fascinate me and in the early 2000s I started to work on language barriers in multinational companies, the topic of a later blogpost.

English-language instruments obscure national differences

In the early 2000s, on a long flight back from a conference – one of the best times to come up with quirky new research ideas - I was wondering whether the language of a questionnaire might have an impact on how people responded. Since survey research was very popular in International Business, this seemed a pertinent question, which - it turned out - nobody in the field had studied before. As I was fortunate enough to have good contacts in many countries, I was able to enlist collaborators in 24 different countries to collect data from undergraduate students querying them on cultural values and the ideal type of job they preferred. I was particularly proud we were able to complete this project without any research funding.

We published two articles on this project, the first using only the data from the first set of countries and the second using data from all countries. We found that even though there were significant differences between the 24 countries for both the native-language questionnaire and the English-language questionnaire, differences were larger for the native-language questionnaire in all but four of the 68 questions.

• Harzing, A.W.; Maznevski, M.; and country collaborators (2002) The interaction between language and culture: A test of the cultural accommodation hypothesis in seven countries, Language and Intercultural Communication, 2(2): 120-139. Available online... - Publisher’s version
• Harzing, A.W.; and 32 country collaborators (2005) Does the Use of English-language Questionnaires in Cross-national Research Obscure National Differences?, International Journal of Cross-Cultural Management, 5(2): 213-224. Available online... - Publisher’s version

These results have important implications for cross-cultural research and hence our understanding of management across cultures. The globalization of the world economy and the increasing importance of multinational companies has made more and more researchers realize that management theories and concepts developed in one part of the world (usually the USA) might not be applicable across borders. In order to find out which theories and concepts are universally valid and which have to be adapted, cross-national research is necessary. However, when these studies are conducted using an English-language questionnaire, we might mistakenly conclude that differences between countries are rather small or non-significant and that management theories and concepts developed in the USA can be applied across cultures.

Rating vs. ranking and language bias

In a follow-up project with MBA students rather than UG students, we took a more methodological stance and investigated whether a reduction of the language bias could be achieved with larger number of scale points (7 instead of 5) and the use of short scenarios with leadership dilemmas, asking respondent to rank, rather than rate, preferred solutions. We did indeed find this to be the case and also found these methodological adjustments to reduce response bias, the topic of another blogpost.

As before, substantive differences between countries were smaller for the English-language version of the questionnaire items than for the native-language versions, thus suppressing potentially relevant country differences. However, the reduction in variance was smaller for 7-point Likert scales than it was for 5-point Likert scales. We also found that the use of scenarios and the ranking of preferred solutions showed an even better performance with regard to language effects. The reduction in variance for the English-language version was very small indeed and only a quarter of the solutions showed a significant reduction in variance.

• Harzing, A.W.; and 26 collaborators (2009) Rating versus ranking: what is the best way to reduce response and language bias in cross-national research?, International Business Review, 18(4): 417-432. Available online... - Publisher’s version

In a later publication we looked into the language effect for the leadership scenarios in more detail. For the six scenarios describing leadership situations pertaining to rewarding, decision-making, goalsetting, face-saving, conflict-resolving, and empathizing we found almost no significant differences in managerial reactions when using the native language as opposed to English within countries. However, echoing extant research on cross-cultural leadership, where national culture is linked to respondents’ nationality, we found significant differences across countries, corroborating the importance of cultural context for leadership and the persistence of cultural variance.

• Zander, L.; Mockaitis, A.; Harzing, A.W. et al. (2011) Standardization and contextualization: A study of language and leadership across 17 countries, Journal of World Business, 46(3): 296-304. Available online... - Publisher’s version

More language research

Since then my interest in language issues has certainly not subsided. After 2011, I published further articles with co-authors Markus Pudelko, Helene Tenzer, Sebastian Reiche, Ling Eleanor Zhang, and  Shea Fan, which have all been the subject of their own blogposts. In 2016 I conducted my inaugural lecture at Middlesex University on the topic.

• Hablas vielleicht un peu la mia language?
• How to manage multi-lingual teams?
• The many benefits of a shared language in multinationals
• Managing expatriates’ identity: subtle desire, big impact
• Why is learning the host country language important for expatriates?
• Language in International Business: A review and agenda for future research
• Managing Expatriates in China: A Language and Identity Perspective

Drop me a line

Free pre-publication versions of these papers are hyperlinked. If you’d like to have an official reprint for these papers, just drop me an email.