Change CAN Happen in Academia: The Story of Organizational Research Methods

[We’re pleased to welcome authors Dr. Herman Aguinis of George Washington University, Ravi S. Ramani of Purdue University Northwest, and Isabel Villamor of George Washington University. They recently published an article in Organizational Research Methods entitled “The First 20 Years of Organizational Research Methods: Trajectory, Impact, and Predictions for the Future” which is currently free to read for a limited time. Below, they reflect on the growth of Organizational Research Methods and possible future directions for the journal.

ORM_72ppiRGB_powerpointA common viewpoint is that change and innovation is difficult and very slow in academia. If they occur at all, changes are long-drawn and unlikely to alter the status quo substantially. The story of Organizational Research Methods (ORM) proves otherwise. ORM, a journal that is just 20 years old, has become one of the most-cited and influential journals in management, business, and applied psychology. How did this happen? And, having achieved so much success so quickly, what does the future of ORM, and methodology more generally, look like?

Our article published in ORM titled “The First 20 Years of Organizational Research Methods: Trajectory, Impact, and Predictions for the Future” answers these questions and more. In two decades, this journal devoted to methodology has fulfilled its dual role and mission of serving as an outlet in which methodologists can publish their best work and where substantive researchers can learn about new methodological developments as well as recommendations on how to address important methodological challenges. From its adoption of a legitimization strategy through strategic partnerships, to growing pains as it sought to balance quantitative vs. qualitative and micro vs. macro topics, to the challenges of breaking into lists of “A-journals,” and finally, to questions about its future, our analysis shows that in many ways, the story of ORM is the story of a successful disruptive new venture in one of the oldest and most traditional industries: academia. We analyze the story of this new venture, as evidenced by editorials, published articles, and the composition of senior editorial teams to understand what specific steps allowed it to succeed. We also highlight innovations introduced by ORM that separated it from other journals, and the researchers whose contributions fueled this rise. Finally, we discuss the implications of ORM’s journey for its future and the future of research methodology as it moves from a growth phase to maturity in its organizational life-cycle.

We believe that our article explicating the trajectory, impact, and possible future directions for ORM and methodology more generally will be useful for management researchers in a number of ways. The information regarding methodological advancements published in ORM will help substantive researchers sharpen their toolkits and discover novel ways of addressing important research questions. It will also help universities, professional organizations, and faculty involved in doctoral education improve the rigor and breadth of training provided to future scholars. Our article can also be a reference to these newcomers as they learn where to go to find accurate answers to most of the methodological questions they may encounter during their formative years. In addition, illustrating the impact of the numerous how-to’s and best-practice articles published in ORM may aid academics who wish to avoid engaging in questionable research practices (QRPs) which damage the credibility and impact of our research. Finally, by showcasing ORM’s trajectory, our article may be of use to the editors and senior editorial teams of both new journals, as well as those interested in improving the impact and influence of their existing publication.

We look forward to hearing the reactions to our article and hope that it will serve as a catalyst further enhance the quality of ORM, and more broadly, methodology in management, business, applied psychology, and related fields.

To read more examples of high impact articles from ORM see this list:

Stay up-to-date with the latest research and sign up for email alerts today through the homepage!

Text Mining in Organizational Research

text-mining-1476780_1920[We’re pleased to welcome authors Dr. Stefan Mol, Vladimer B. Kobayashi, Hannah A. Berkers, Gabor Kismihok, and Deanne N. Den Hartog of the University of Amsterdam. They recently published an article in Organizational Research Methods entitled “Text Mining in Organizational Research,” which is currently free to read for a limited time. Below, Dr. Mol recounts the events that led to the research and the significance it has to the field:]

07ORM13_Covers.inddWere there any specific external events that influence your decision to pursue this research?
One critical on-going event that lead us to pursue this research is the revolution and promise brought by the rise of big data to understand and enhance organizational processes. A large proportion of these data are comprised of texts that are generated every day at rates that imply that manual analysis of all of this data is no longer possible. The abundance of untapped text data suggest the existence of information with the promise of generating new knowledge that may be used to enhance both individual and organizational level outcomes.
Although, organizations already collect and store text data, many do not fully take advantage of the knowledge that can be gleaned from analyzing text. This may be due to a lack of expertise in conducting automatic text analysis or text mining. The mission of our work here is to empower organizational researchers by raising awareness of the possibilities afforded by text mining, helping them see how text mining might help them answer their research questions, and helping them to understand and use the text mining process and tools.
In what ways is your research innovative, and how do you think it will impact the field?
With this article we hope to contribute by facilitating dialogue between data scientists and organizational researchers about the opportunities afforded by text mining. As an example, we illustrate the role that text mining of vacancies might play in job analysis. Previous approaches to job analysis rely on time consuming collection and analysis of survey and observation based data the results of which soon become outdated due to the fast changing nature of jobs. Using text mining we demonstrate how one can take advantage of other data sources such as online job vacancies to understand the requirements and skill demands of different types of jobs. Our goal is to not only apply text mining to the field of job analysis but more importantly to inform organizational researchers about the wide-ranging uses text mining could have in organizational research. We hope that this will spark an increase in the use of text data and machine learning in organizational research.
What advice would you give to new scholars and incoming researchers in this particular field of study?
Existing text mining solutions are technique and tool-oriented because most techniques and Big Data tools are currently primarily shaped by technical fields, such as statistics and computer science, that put greater emphasis on the computational and technological aspects. However, applying these in the field of organizational research holds great promise. Organizational researchers bring with them a repertoire of organizational theories. These theories provide domain specific information and requirements that can influence the selection of techniques and analytical strategy, and the way to evaluate the success of the particular application. Our advice for incoming organizational researchers wanting to explore text mining is to draw on their own theoretical expertise and from there start selecting the appropriate techniques and approaches to text mining. Also, as with using other analytical tools, we do need to pay careful attention to rigor in evaluation and validation of text mining based results.

Stay up-to-date with the latest research from Organizational Research Methods and sign up for email alerts today through the homepage!




An Argument for Compassionate Research Methods

9505520762_1ec974cdf1_z[We are pleased to welcome Hans Hansen of Texas Tech University. Hans recently published an article in Organizational Research Methods, entitled “This is Going to Hurt: Compassionate Research Methods” with co-author Christine Quinn Trank of Vanderbilt University.]

Compassionate research hopes to make the world a better place by reducing suffering, but it can also provide our field with new theories, which we desperately need. When you look at the world with a new lens, you see new things, things that other lenses could not reveal. We hope that a compassionate approach can not only reveal new aspects of existing phenomena, but entirely new phenomena as well, and lead to entirely new theories of organizing.

Current Issue Cover

The topic of compassion is making an impact in organizational studies, and interest continues to increase, so our aim was to provide a methodology for this burgeoning field. In addition to moving us in new directions, we also hope to increase compassionate research by clearing outlining a distinct method.

We hope to give the field a push, and just as grounded theory provided a clear method for inductive research, we hope compassionate methods become the guide for compassionate research, and be generative in providing new insights and theories.

The abstract for the paper:

As compassion has become established in the organizational literature as an important area of study, calls for increased compassion in our own work and research have increased. Compassion can take many forms in academic work, but in this article we propose a framework for compassionate research methods. Not only driven by caring for others and a desire for improving their lot, compassionate research methods actually immerse the researcher in compassionate work. We propose that compassionate research methods include three important elements: ethnography, aesthetics, and emotionality. Together, these provide opportunities for emergent theoretical experimentation that can lead to both the alleviation of suffering in the immediate research context and new theoretical insights. To show the possibilities of this method, we use empirical data from a unique setting—the first U.S. permanent death penalty defense team.

You can read “This is Going to Hurt: Compassionate Research Methods” from Organizational Research Methods free for the next two weeks by clicking here. Want to keep current on all of the latest research from Organizational Research MethodsClick here to sign up for e-alerts!

*Conversation image attributed to Andreas Bloch (CC)

Top Five Articles from Organizational Research Methods

JPG READINGSummer is just around the corner, bringing with it longer days and warmer weather. To celebrate the season, we present a list of most read articles from Organizational Research Methods to add to your summer reading list.

“Seeking Qualitative Rigor in Inductive Research: Notes on the Gioia Methodology” by Dennis A. Gioia, Kevin G. Corley, and Aimee Hamilton (January 2013)

For all its richness and potential for discovery, qualitative research has been critiqued as too often lacking in scholarly rigor. The authors summarize a systematic approach to new concept development and grounded theory articulation that is designed to
bring “qualitative rigor” to the conduct and presentation of inductive research.

Current Issue Cover

“Validation of a New General Self-Efficacy Scale” by Gilad Chen, Stanley M. Gully, and Dov Eden (January 2001)

Researchers have suggested that general self-efficacy (GSE)can substantially contribute to organizational theory, research, and practice. Unfortunately, the limited construct validity work conducted on commonly used GSE measures has highlighted such potential problems as low content validity and multidimensionality. The authors developed a new GSE (NGSE) scale and compared its psychometric properties and validity to that of the Sherer et al. General Self-Efficacy Scale (SGSE). Studies in two countries found that the NGSE scale has higher construct validity than the SGSE scale. Although shorter than the SGSE scale, the NGSE scale demonstrated high reliability, predicted specific self-efficacy (SSE) for a variety of tasks in various contexts, and moderated the influence of previous performance on subsequent SSE formation. Implications, limitations, and directions for future organizational research are discussed.

“Common Beliefs and Reality About PLS: Comments on Rönkkö and Evermann (2013)” by Jörg Henseler, Theo K. Dijkstra, Marko Sarstedt, Christian M. RingleAdamantios Diamantopoulos, Detmar W. Straub, David J. Ketchen Jr.Joseph F. Hair, G. Tomas M. Hult, and Roger J. Calantone (April 2014)

This article addresses Rönkkö and Evermann’s criticisms of the partial least squares (PLS) approach to structural equation modeling. We contend that the alleged shortcomings of PLS are not due to problems with the technique, but instead to three problems with Rönkkö and Evermann’s study: (a) the adherence to the common factor model, (b) a very limited simulation designs, and (c) overstretched generalizations of their findings. Whereas Rönkkö and Evermann claim to be dispelling myths about PLS, they have in reality created new myths that we, in turn, debunk. By examining their claims, our article contributes to reestablishing a constructive discussion of the PLS method and its properties. We show that PLS does offer advantages for exploratory research and that it is a viable estimator for composite factor models. This can pose an interesting alternative if the common factor model does not hold. Therefore, we can conclude that PLS should continue to be used as an important statistical tool for management and organizational research, as well as other social science disciplines.

“Using Generalized Estimating Equations for Longitudinal Data Analysis” by Gary A. Ballinger (April 2004)

The generalized estimating equation (GEE) approach of Zeger and Liang facilitates analysis of data collected in longitudinal, nested, or repeated measures designs. GEEs use the generalized linear model to estimate more efficient and unbiased regression parameters relative to ordinary least squares regression in part because they permit specification of a working correlation matrix that accounts for the form of within-subject correlation of responses on dependent variables of many different distributions, including normal, binomial, and Poisson. The author briefly explains the theory behind GEEs and their beneficial statistical properties and limitations and compares GEEs to suboptimal approaches for analyzing longitudinal data through use of two examples. The first demonstration applies GEEs to the analysis of data from a longitudinal lab study with a counted response variable; the second demonstration applies GEEs to analysis of data with a normally distributed response variable from subjects nested within branch offices ofan organization.

“Answers to 20 Questions About Interrater Reliability and Interrater Agreement” by James M. LeBreton and Jenell L. Senter (October 2008)

The use of interrater reliability (IRR) and interrater agreement (IRA) indices has increased dramatically during the past 20 years. This popularity is, at least in part, because of the increased role of multilevel modeling techniques (e.g., hierarchical linear modeling and multilevel structural equation modeling) in organizational research. IRR and IRA indices are often used to justify aggregating lower-level data used in composition models. The purpose of the current article is to expose researchers to the various issues surrounding the use of IRR and IRA indices often used in conjunction with multilevel models. To achieve this goal, the authors adopt a question-and-answer format and provide a tutorial in the appendices illustrating how these indices may be computed using the SPSS software.

All of the above articles from Organizational Research Methods will be free to access for the next two weeks. Want to know all about the latest research from Organizational Research Methods? Click here to sign up for e-alerts!

*Reading image attributed to Herry Lawford (CC)

The Benefits of Establishing Causal Order in Longitudinal Studies

9423385629_171671f9c2_z[We are pleased to welcome Gerhard Kling of University of London. Gerhard recently published an article in Organizational Research Methods, entitled “Establishing Causal Order in Longitudinal Studies Combining Binary and Continuous Dependent Variables” with co-authors Charles Harvey of Newcastle University Business School and Mairi Maclean of Newcastle University Business School.]

  • What inspired you to be interested in this topic?

We always wonder what comes first, a change in management practice or performance. Are successful companies successful because they were successful in the past? Or did changes in management cause their success?

  • Were there findings that were surprising to you?

Initially, we did not intend to develop a new method, the QSP-VAR (qualitative short panel vector autoregression). The reviewers pushed us into this direction, which led to the development of the QSP-VAR. The surprising fact is that our method can be applied in any area of research where there are binary and continuous dependent variables (e.g. genetics).07ORM13_Covers.indd

  • How do you see this study influencing future research and/or practice?

As we provide our code (OpenBugs), we hope that other researchers can use the QSP-VAR in their research. We believe that there is a lot of potential in areas where causal order between variables is unknown.

The abstract:

Longitudinal studies with a mix of binary outcomes and continuous variables are common in organizational research. Selecting the dependent variable is often difficult due to conflicting theories and contradictory empirical studies. In addition, organizational researchers are confronted with methodological challenges posed by latent variables relating to observed binary outcomes and within-subject correlation. We draw on Dueker’s qualitative vector autoregression (QVAR) and Lunn, Osorio, and Whittaker’s multivariate probit model to develop a solution to these problems in the form of a qualitative short panel vector autoregression (QSP-VAR). The QSP-VAR combines binary and continuous variables into a single vector of dependent variables, making every variable endogenous a priori. The QSP-VAR identifies causal order, reveals within-subject correlation, and accounts for latent variables. Using a Bayesian approach, the QSP-VAR provides reliable inference for short time dimension longitudinal research. This is demonstrated through analysis of the durability of elite corporate agents, social networks, and firm performance in France. We provide our OpenBUGS code to enable implementation of the QSP-VAR by other researchers.

You can read “Establishing Causal Order in Longitudinal Studies Combining Binary and Continuous Dependent Variables” from Organizational Research Methods for free by clicking here. Want to know all about the latest research from Organizational Research MethodsClick here to sign up for e-alerts!

*Numbers picture credited to morebyless (CC)


Gerhard KlingGerhard Kling is professor of international business and management at SOAS University of London. He studied economics and mathematics and holds a PhD in economics from the University of Tuebingen. His research focuses on quantitative research methods with applications in management and finance. His recent publications are in Organization Studies, the British Journal of Management, and the International Journal of Research in Marketing.

Charles Harvey is professor of business history and management at Newcastle University Business School. He holds a PhD in international business from the University of Bristol. He is author of numerous books and articles in the fields of strategy, leadership, and management. His research focuses on the historical processes that inform contemporary business practice, entrepreneurial philanthropy, and the exercise of power by elite groups in society. His recent publications are in the Academy of Management Review, Organization Studies, Organization, Human Relations, and the Business History Review.

Mairi Maclean is professor of international management and organization studies at Newcastle University Business School, where she is director of research. She received her PhD from the University of St Andrews. Her research interests include international business elites and elite power, entrepreneurial philanthropy, and history and organization studies. She is the author of four books, including Business Elites and Corporate Governance in France and the UK (Palgrave Macmillan, 2006) with Charles Harvey and Jon Press, and editor of a further four. Recent publications include contributions to the Academy of Management Review, Organization Studies, Human Relations, Organization, and the Business History Review.

Call for Papers on Neuroscience in Organizational Research

brainy-people-1072657-mOrganizational Research Methods seeks submissions for a feature topic on Neuroscience in Organizational Research. This feature topic will be guest edited by Micah Murray and John Antonakis, both of Lausanne University.

From the call for papers:

In many areas of the social and behavioral sciences, neuroscience has emerged as one of the dominant conceptual and methodological frameworks for studying human behavior. Although it originally gained traction in the psychological sciences, the 07ORM13_Covers.inddneuroscience paradigm has since spread to other areas in the social sciences including economics, marketing, and finance. However, with a few notable exceptions, researchers in management and applied psychology have been slow to embrace neuroscientific models and methods (for a few illustrative exceptions see Bagozzi et al., 2013; Balthazard, Waldman, Thatcher, & Hannah, 2012). One explanation for this reticence, may be that researchers lack an appreciation for the diversity of neuroscience methods that are available and how these methods might be incorporated into their science.

The purpose of this feature topic is threefold. First, we intend to expose organizational scholars to the broad array of neuroscience methods and how these methods might be used to test substantive research questions (both basic and applied). Second, we intend to provide illustrative examples that empirically demonstrate the value-added nature of these methods. Finally, because no method or set of methods are without limitations, we intend to provide critical reviews of these methods so that their strengths and limitations may be better understood by organizational scholars.

Organizational Research Methods will be publishing a two-part Feature Topic devoted to Neuroscience in Organizational Research. The first part will consist of invited papers while the second part consists of a call for papers that will extend what is presented in Part I. Proposals of no more than 5 pages double-spaced should be emailed to both guest editors anytime prior to September 30, 2015. For more information, including topics which have been commissioned for Part 1 and contact information, click here.

Want to know about all the latest news and research from Organizational Research Methods? Click here to sign up for e-alerts!

Organizational Research Methods Announces the Winner of the 2013 Best Paper Award!

Steve W. J. Kozlowski, Georgia T. Chao, James A. Grand, Michael T. Braun and Goran Kuljanin are thistrophy-189659-m year’s winners of the 2013 Best Paper Award from Organizational Research Methods! The scholars were honored at the Academy of Management 2014 Annual Meeting for their article entitled “Advancing Multilevel Research Design: Capturing the Dynamics of Emergence” which appeared in the October 2013 issue of Organizational Research Methods and can be read for free for the next 30 days!

The abstract:

Multilevel theory and research have advanced organizational science but are limited because the research focus is incomplete. Most quantitative research examines top-down, contextual, cross-level relationships. Emergent phenomena that manifest from the bottom up from the psychological characteristics, processes, and interactions 07ORM13_Covers.inddamong individuals—although examined qualitatively—have been largely neglected in quantitative research. Emergence is theoretically assumed, examined indirectly, and treated as an inference regarding the construct validity of higher level measures. As a result, quantitative researchers are investigating only one fundamental process of multilevel theory and organizational systems. This article advances more direct, dynamic, and temporally sensitive quantitative research methods designed to unpack emergence as a process. We argue that direct quantitative approaches, largely represented by computational modeling or agent-based simulation, have much to offer with respect to illuminating the mechanisms of emergence as a dynamic process. We illustrate how indirect and direct approaches can be complementary and, appropriately integrated, have the potential to substantially advance theory and research. We conclude with a set of recommendations for advancing multilevel research on emergent phenomena in teams and organizations.

Click here to read “Advancing Multilevel Research Design: Capturing the Dynamics of Emergence” for free from Organizational Research Methods! Like what you read? Click here to sign up for e-alerts and get all the latest research from Organizational Research Methods sent straight to your inbox!