What if fully agree doesn't mean the same thing across cultures?

Explains how response style difference between countries can distort cross-national comparisons

Anne-Wil Harzing - Thu 2 Feb 2017 08:55 (updated Sun 24 Apr 2022 10:55)

agree

Studies of attitudes across countries generally rely on a comparison of aggregated mean scores to Likert-scale questions. This presupposes that when people complete a questionnaire, their answers are based on the substantive meaning of the items to which they respond. However, people's responses are also influenced by their response style. Hence, the studies we conduct might simply reflect differences in the way people respond to surveys, rather than picking up real differences in phenomena across countries.

What are response styles?

Response styles refer to a respondent’s tendency to systematically respond to questionnaire items regardless of item content. The main type of response styles are:

Acquiescent Response Style (ARS) where respondents are more likely to agree or give a positive response to a question
Dis-acquiescent Response Style (DRS) where respondents are more likely to disagree or give a negative response to a question
Extreme Response Style (ERS) where the response is more likely to be highly positive or highly negative
Middle Response Style (MRS) where there is a greater tendency to go for an ‘average’ response.

High ARS implies better/higher scores while ERS gives you more varied or extreme (and possibly higher) scores than MRS.

Study 1: Response style differences in 26 countries

My 2006 study [which became the most cited article in the International Journal of Crosscultural Management, the journal it was published in, since its inception] showed that there are major differences in response styles between countries. Country-level characteristics such as power distance, collectivism, uncertainty avoidance and extraversion all have a positive influence on response styles such as acquiescence and (positive) extreme response styles.

There are also clear country clusters. Northern and Western European countries show lower levels of acquiescence and higher levels of dis-acquiescence, whereas the reverse is true for Southern European and most Latin American countries, with France displaying a “bridgehead” position. Within Eastern Europe two clear patterns are visible with Russia and Poland showing high dis-acquiescence, low MRS and low positive ERS, while Bulgaria and Lithuania show the reverse pattern.

Respondents in China and Hong Kong show medium acquiescence, low dis-acquiescence, low positive and negative ERS and high MRS, while Malaysia and India show the reverse pattern on nearly all of these indicators. Taiwan takes up a middle position between these extremes. A third and very distinct pattern is shown by Japan, which has the lowest acquiescence, the highest dis-acquiescence and the highest MRS of all 26 countries. The results for Malaysia show that ethnic background also influences response styles: Malaysian-born Chinese respondents had response styles that differed significantly from Malay respondents and were generally situated between Malay and Chinese (mainland China & Hong Kong) in terms of response styles.

This article resulted in quite a bit of interest from consultancy companies working in the area of employee surveying. One of them even interviewed me for a post on their website.

languages

Study 2: Ranking and language

A second study (Harzing et al. 2009) based showed that even though changing from a 5-point Likert scale to a 7-point Likert scale reduced MRS and ERS, it did not fully eliminate the problem of differences between countries in their tendency to use middle or end points of the scale. As expected, switching to a 7-point scale did very little to address the major differences between countries in terms of DRS and ARS. However, our results suggest that ranking – asking respondents to rank responses in order of preference - provides an excellent alternative, as it completely eliminates both MRS/ERS and ARS/DRS.

Our second study also showed that substantive differences between countries are smaller for the English-language version of the questionnaire items than for the native-language versions, thus suppressing potentially relevant country differences. However, the reduction in variance was smaller for 7-point Likert scales than it was for 5-point Likert scales. The reduction in variance for the English-language version was very small indeed, and only a quarter of the solutions showed a significant reduction in variance.

Study 3: Scale anchors and prior subject knowledge

A third study included five groups of respondents: Chinese in China, Chinese in Australia, Anglo-Australians in Australia, and two groups of German students in Germany. We asked them to indicate on a 10-point scale whether certain employee attitudes or behaviour were more typically Australian (left-hand of the scale) or Chinese (right-hand of the scale). We then asked them how they would rate the performance (low to high on a 10-point scale) of an employee who displayed this attitude or behaviour.

When scale anchors referred to naturally opposing and mutually exclusive constructs (Australian vs. Chinese) respondents showed more ERS than when they referred to level or degree of a construct (low-high performance). A higher level of knowledge of cross national differences through prior education also resulted in higher ERS on behavioural questions – probably reflecting the higher levels of confidence in the answers given – but not on performance questions.

knowledge

Conclusion and references

Although there are some remedies to eliminate or alleviate response bias, the first step towards finding a solution is acknowledging that response bias can be a serious threat to valid comparisons across countries. We hope this blog has provided a step in that direction and that in future response bias will receive the attention it deserves from researchers in the area of international and cross cultural management.

Harzing, A.W. (2006) Response styles in cross-national mail survey research: A 26-country study, The International Journal of Crosscultural Management, 6(2): 243-266. [reprinted in Cross-Cultural Management (in the Routledge series Critical Perspectives on Business and Management) Editors: Tim G. Andrews and Richard Mead, Routledge, 2008]. Most cited article (in Scopus) in IJCCM since its inception. Available online... - Interview by Deep Insight based on this article - Publisher’s version
Harzing, A.W.; and 26 collaborators (2009) Rating versus ranking: what is the best way to reduce response and language bias in cross-national research?, International Business Review, 18(4): 417-432. Available online... - Publisher’s version
Harzing, A.W.; Brown, M.; Köster, K., Zhao, S. (2012) Response style differences in cross-national research: dispositional and situational determinants, Management International Review, 52(3): 341-363. Available online... - Publisher's version (read for free)

JIBS articles on methodological issues

[From the AIB SIG Research Methods website: https://rmsig.aib.world/resources-2/jibs-methods-editorials/]

Drop me a line

Free pre-publication versions of these papers are hyperlinked. If you’d like to have an official reprint for these papers, just drop me an email.

research focus response styles international business international research language response rates research methods

Anne-Wil Harzing is Emerita Professor of International Management at Middlesex University, London. She is a Fellow of the Academy of International Business, a select group of distinguished AIB members who are recognized for their outstanding contributions to the scholarly development of the field of international business. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.

Anne-Wil Harzing's profile and contact details >>