The Search for Cross-level Interaction Effects: Fruitful or Futile?


[We’re pleased to welcome Herman Aguinis of Indiana University and Steven Andrew Culpepper of the University of Illinois at Urbana-Champaign. Drs. Aguinis and Culpepper recently collaborated on their article from Organizational Research Methods entitled “An Expanded Decision-Making Procedure for Examining Cross-Level Interaction Effects With Multilevel Modeling.”]

Researchers in organizational behavior, human resource management, 07ORM13_Covers.inddentrepreneurship, strategy, sociology, psychology, education, and many other fields now explicitly recognize that lower-level entities are usually nested within higher-level collectives. For example, employees are nested within jobs and teams, establishments within companies, and firms within industries. However, how do we know whether there is variability of a lower-level relationship across higher-order units? The answer to this question has important implications in terms of the appropriateness of using usual data-analytic techniques based on ordinary least squares regression, or whether a multilevel modeling approach should be used. Moreover, the presence of variability in lower-order relationships across higher-order units leads to an examination of potential contextual factors that may serve as moderators of such relationships. In addition, practitioners are particularly interested in such effects because they provide information on the contextual conditions and processes under which interventions focused on individuals (e.g., selection, leadership training, performance appraisal and management) result in more or less positive outcomes.

Our article which appears in Organizational Research Methods titled “An Expanded Decision Making Procedure for Examining Cross-level Interaction Effects with Multilevel Modeling” offers a new index to assess variability of lower-level relationships across higher-order processes and units. This new index is labeled intraclass correlation beta (i.e., ρ_β). We illustrate the computation of ρ_β using previously published articles and also a Monte Carlo study. Our results suggest that researchers contemplating the use of multilevel modeling, as well those who suspect nonindependence in their data structure, should expand the decision criteria for using multilevel approaches to include ρ_β. To facilitate this process, we offer illustrative data sets and the icc_beta R package for computing ρ_β in single and multiple-predictor situations and make them available through the Comprehensive R Archive Network (i.e., CRAN).

We are very excited about the potential of ρ_β to allow us to uncover the presence of variability in lower-order relationships across higher-order process and units and look forward to discoveries that can be made based on information provided by ρ_β. Moreover, ρ_β can also be used as an index of effect size and used to synthesize previously published research to understand which may be more or less fruitful research domains in which cross-level moderating effects may exist.

You can read “An Expanded Decision-Making Procedure for Examining Cross-Level Interaction Effects With Multilevel Modeling” from Organizational Research Methods free for the next week by clicking here. Want to know about all the latest research from Organizational Research Methods? Click here to sign up for e-alerts!

aguinis5Herman Aguinis ( is the John F. Mee Chair of Management and Founding Director of the Institute for Global Organizational Effectiveness in the Kelley School of Business, Indiana University. His multi-disciplinary, multi-method, and multi-level research addresses human capital acquisition, development, and deployment, and research methods and analysis. He has published five books and more than 120 articles in refereed journals. He is a Fellow of the Academy of Management, former editor-in-chief of Organizational Research Methods, and received the 2012 Academy of Management Research Methods Division Distinguished Career Award for lifetime contributions.

Steven Andrew Culpepper ( is an assistant professor in the Department of Statistics at the University of Illinois at Urbana-Champaign. He completed a doctorate in educational psychology from the University of Minnesota in 2006. His research focuses on statistical methods in the social sciences and includes the development of new methodologies, evaluation of existing procedures, and application of novel statistical techniques to substantive questions in demography, education, management, and psychology.

Tags: , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: