The ins and outs of experimental research in IB and Management

Reports on two professional development workshops on experimental research at the 2021 AIB and IACMR conferences

Carys Chan - Sat 23 Oct 2021 15:25 (updated Thu 2 Jun 2022 12:03)

[Guest post by CYGNA member Carys Chan. Carys reports on two conference workshops on experimental design that were co-organized with Shea Fan and  Kähäri Perttu.]

Experimental design is a powerful research method that can provide causal evidence and improve the internal validity of the results, but it is not a mainstream research method in International Business (IB) or International Management (IM) research.

An editorial published by the editors of Journal of International Business Studies (JIBS), the leading IB outlet, identified only eight articles using this method out of over 900 empirical articles, that is fewer than 1% of all published empirical research in JIBS. In a review of language research in IB in 2020, Fan and Harzing (2020) only found eight out of 300 publications adopting an experimental design.

It certainly does appear that experiments are underrepresented and underappreciated in IB/IM research. In contrast, there is renewed interest in the use of experimental designs in management and leadership research, with some referring to them as the “gold standard” of scientific research. Approximately 11.5% of articles published in 2015-2018 used experimental design up from only 7% in 1990s (Podsakoff & Podsakoff, 2019).

Two virtual professional development workshops

At the 2021 AIB annual conference and the 9^th Biennial International Association of Chinese Management Research (IACMR) conference, Shea Fan, Carys Chan and  Perttu Kähäri invited leading IB and management scholars to share their experiences and insights on using a variety of experimental designs in IB and management research.

Both virtual professional development workshops (PDWs) were very well received and well-attended by early career scholars from around the world. We compiled the event video, slides, and published articles in this post with the aim of helping researchers who are interested to use experimental designs in their research. Read on to learn more!

Academy of International Business

• Introduction: Shea Fan (00:00-08:40). Download slides
• Professor David Thomas: Thinking experimentally (08:40-15:10). Download slides
• Professor Paula Caligiuri: Experiments in organizations (15:40-23:55). Download slides
• Professor Melody Chao: Promoting knowledge accumulation: the role of experiments (24:20 – 36:40). Download slides

• Professor Margaret Shaffer: Using judgement tasks to understand decisions in expatriation (36:45-48:00). Download slides
• Professor Ronald Fischer: Don't neglect the individual – the power of within-subject designs (48:15-58:50). Download slides
• Discussions moderate by Perttu Kähäri: (58:50-1:11:50)

Research design examples

• Multi-study design, survey and experiment (1:08:29): Brienza, J. P., Kung, F. Y., & Chao, M. M. (2021). Wise reasoning, intergroup positivity, and attitude polarization across contexts. Nature communications, 12(1), 1-11. https://doi.org/10.1038/s41467-021-23432-1

• Using a within-subject design in cross-cultural management: Fischer, R., & Karl, J. A. (2020). Experimental methods in cross-cultural management. In B. Szkudlarek; L. Romani ; D. Caprar; J.S. Osland (Eds.), The SAGE handbook of cross-cultural management, 111-126. Download from here.

• Example on quasi-experimental within-subject design: Singh, P., Tewari, S., Kesberg, R., Karl, J. A., Bulbulia, J., & Fischer, R. (2020). Time investments in rituals are associated with social bonding, affect and subjective health: a longitudinal study of Diwali in two Indian communities. Philosophical Transactions of the Royal Society B, 375(1805), 20190430, https://doi.org/10.1098/rstb.2019.0430

International Association of Chinese Management Research

Role and purpose of experimental research

Professor Roy Chua kicked off the professional development workshop by explaining the roles and purposes of using experiments to advance theory and practice in management research, and exploring different ways an experimental design can be incorporated into management research. Download slides

• Research design example: Multi-study design, survey and experiment: Chua, R. Y., Morris, M. W., & Mor, S. (2012). Collaborating across cultures: Cultural metacognition and affect-based trust in creative collaboration. Organizational Behaviour and Human Decision Processes, 118(2), 116-131. https://doi.org/10.1016/j.obhdp.2012.03.009

Increasing validity of your experimental design

Professor Ke Michael Mai discussed different types of experimental designs and shared how we can bring naturalistic manipulation into the experiment to enhance the validity of the manipulation, and also design the experiment to mirror a more naturalistic setting (i.e., the field). Multiple experimental design examples were included in the slides. Download slides.

• Research design example: Experimental design using a naturalistic setting: Wu, S. J., & Paluck, E. L. (2021). Designing nudges for the context: Golden coin decals nudge workplace behavior in China. Organizational Behavior and Human Decision Processes, 163, 43-50. https://doi.org/10.1016/j.obhdp.2018.10.002

Priming as a form of manipulation

Professor Chi-Ying Cheng provided an overview of priming as methods for measuring culture and used examples from her own and other’s research to illustrate the types of cultural priming. In the articles below, you can see how Professor Cheng used priming as a form of manipulation in her experimental studies:

• Cheng, C.-Y. & Hong, Y.-y. (2017). Kiasu and creativity in Singapore: An empirical test of the situated dynamics framework. Management and Organizational Review. 13(4), 871-894. doi:10.1017/mor.2017.41

• Cheng, C.-Y., K. Hanek, A. Odom, & Lee, F. (2021). Divided loyalties: Identity integration and cultural cues predict ingroup favoritism among biculturals. International Journal of Intercultural Relations, 80, 321-335. DOI: 10.1016/j.ijintrel.2020.10.003

• Cheng, C.-Y. & Leung, A. K.-y. (2013). Revisiting the Multicultural Experience—Creativity Link: The Effects of Perceived Cultural Distance and Comparison Mindset. Social and Personality Psychological Science, 4(4), 475-482. doi.org/10.1177/1948550612462413

Common mistakes and challenges

Finally, we also had the privilege to learn from Professor Zhi Liu who shared her personal experiences and observations of common mistakes and mis-perceptions of experiments, as well as the difficulties of designing experiments to examine workplace phenomena such as leadership, team, and HR decision-making. An example of using vignettes as manipulation:

• Siegel, P. A., Brockner, J., Wiesenfeld, B. M., & Liu, Z. (2016). Non-contingent success reduces people’s desire for processes that adhere to principles of fairness. Social Justice Research, 29(4), 375-401. https://doi.org/10.1007/s11211-016-0272-z

Related videos

Related blogposts

• Research Methods in IB – Trends and Future Agenda
• AIB RM-SIG: Research methods in International Business
• Challenges in International survey research: illustrations and solutions
• Language effects in international mail surveys
• What if fully agree doesn't mean the same thing across cultures?
• Should we distance ourselves from the cultural distance concept?
• Experimental research in international management