The Chrysalis Effect: Publication Bias in Management Research

14523043285_2235b0dbb4_zHow well do published management articles represent the broader management research? To say that questionable research practices impact only a few articles ignores the broader, systemic issue effecting management research. According to authors Ernest Hugh O’Boyle Jr., George Christopher Banks, and Erik Gonzalez-Mulé, the high pressure for academics to publish leads many to engage in questionable research, thereby leading the resulting published articles to be biased and unrepresentative. In their article, “The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles,” published in Journal of Management, O’Boyle, Banks, and Gonzalez-Mulé delve into the issue of questionable research practices. The abstract for the paper:

The issue of a published literature not representative of the population of research is Current Issue Covermost often discussed in terms of entire studies being suppressed. However, alternative sources of publication bias are questionable research practices (QRPs) that entail post hoc alterations of hypotheses to support data or post hoc alterations of data to support hypotheses. Using general strain theory as an explanatory framework, we outline the means, motives, and opportunities for researchers to better their chances of publication independent of rigor and relevance. We then assess the frequency of QRPs in management research by tracking differences between dissertations and their resulting journal publications. Our primary finding is that from dissertation to journal article, the ratio of supported to unsupported hypotheses more than doubled (0.82 to 1.00 versus 1.94 to 1.00). The rise in predictive accuracy resulted from the dropping of statistically nonsignificant hypotheses, the addition of statistically significant hypotheses, the reversing of predicted direction of hypotheses, and alterations to data. We conclude with recommendations to help mitigate the problem of an unrepresentative literature that we label the “Chrysalis Effect.”

You can read “The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles” from Journal of Management free for the next two weeks by clicking here.

Want to stay current on all of the latest research published by Journal of ManagementClick here to sign up for e-alerts! You can also follow the journal on Twitter–read through the latest tweets from Journal of Management by clicking here!

*Library image attributed to Apple Vershoor (CC)


Stories of Research to Reality: Deborah Rupp on Fairness in the Workplace

In early May SAGE gathered seven social scientists on Capitol Hill to tell stories, stories of their discipline’s impact on society and the economy, and stories of their own academic journey. The underlying goal of “Stories of Research to Reality: How the Social Sciences Change the World” was both to mark SAGE’s 50th birthday as an independent publisher and to demonstrate the value and impact of social science itself, increasingly under attack as either a waste or a luxury by some legislators.

Deborah Rupp (2)

The entire event, moderated by prominent blogger and George Washington University political scientist John Sides and held at the Hart Senate Office Building, was recorded; the seven individual videos are being published over the next seven weeks.  Each tale presents one facet of the real-world value of actual social and behavioral science research, with the implicit message that this is scholarship we should be encouraging.

The first speaker in this series is Deborah E. Rupp, a management professor at Purdue University and the William C. Byham Chair in Industrial/Organizational Psychology.

She conducts research on organizational justice, behavioral ethics, corporate social responsibility, and humanitarian work psychology; as well as issues surrounding behavioral assessment, technology, bias, and the law. Her research has been cited in U.S. Supreme Court proceedings, and she has worked with organizations around the world such as UNICEF, the Emirates Group, and the South Korean government. Rupp has published three books and more than 80 papers and chapters, and is the former editor-in-chief of Journal of Management, the top-ranked empirical journal in business, management, and applied psychology.

In her talk, Rupp describes  some of the specific areas she’s taken her discipline — industrial organization psychology, which is the scientific study of human behavior in the workplace – such as exploring aging in the workplace, recruiting and training returning veterans for civilian fields, and the psychological impact of being unemployed. She then zeroes in on her current research, studying unfairness (or the perception of unfairness) on the job.

Employees who believe their job is just are happier, healthier and more productive, while those who field aggrieved are sicker, less productive and prone to introduce unwanted behaviors from theft to litigiousness. “People,” she explains, “expect other people to treat each other fairly – just because.”

Upcoming speakers in this series include:

Bruce Bueno De Mesquita | Julius Silver Professor of Politics, New York University

Claire M. Renzetti | professor of sociology, University of Kentucky

John W. Creswell | professor of educational psychology, University of Nebraska-Lincoln

Michael Reisch | Daniel Thursz Distinguished Professor of Social Justice, University of Maryland

Jim Knight |research associate, University of Kansas Center for Research on Learning, and director of the Kansas Coaching Project

Kerric Harvey | associate professor of media and public affairs, and associate director of the Center for Innovative Media, George Washington University


Republished with permission. The original post was published on Social Science Space.

Challenges for Institutional Theory

Challenges for Institutional Theory”, by Roy Suddaby, University of Alberta, was the most frequently read article in the Journal of Management Inquiry in 2010. Roy has provided additional background:

The catalyst for this article was a conference on Institutional Theory sponsored by the Australian School of Business at the University of New South Wales in July of 2008 where I was asked to give the keynote address discussing the future of neo-institutionalism in management research. Neo-institutionalism (or institutional theory) is itself, a challenge to traditional economic explanations for firm behaviour in business research. However, it has grown to be incredibly popular and has largely eclipsed traditional economic explanations for firm behaviour, at least within the Organizations and Management Theory division of the Academy of Management.

 My essay, thus, offers a critique of what used to itself be a critique but has grown to become a dominant, if not somewhat hegemonic, perspective on firm behaviour. Because of its popularity, there has been a tendency for management researchers to try to recast their research in institutional terms. The danger of doing this, I argue, is twofold. First, there is a tendency to overlook more simplistic, economically rational explanations for firm behaviour. There is a principle in science – Occam’s razor – that suggests when we have two competing explanations for an event we ought to prefer the simpler and more direct explanation. My essay raises the possibility that management scholars have abandoned this principle, often choosing more tortuous and complicated explanations based in neo-institutionalism, when more obvious explanations exist. Second, as more and more firm behaviour becomes characterized by scholars as ‘institutional behaviour’ there is a danger of trivializing institutional theory. This occurs by treating any form of change, however marginal or inconsequential, as ‘institutional change’ or any act of entrepreneurship as ‘institutional entrepreneurship’. The focus of the theory, however, is to address issues of ‘profound’ change or acts of entrepreneurship that fundamentally redefine market and social transactions.

 The intent of the essay, thus, was to avoid the tendency of management scholarship to follow current fads and fashions and retain some degree of scepticism in our research. I suspect the essay resonated with management scholars and hopefully will accomplish my objective of trying to reign in the relatively flippant treatment of institutionalism while offering an attempted re-articulation of the core purpose of institutional theory and its research agenda. Institutional theory still has a very large and interesting domain of phenomena to explore – particularly now. There is a growing awareness amongst executives of publicly traded corporations that they now compete both in the material world for material resources (inputs, labour, capital) and in the social world for symbolic resources (reputation, legitimacy, status). This is perhaps most apparent in the growing debate over global sustainability and corporate social responsibility. Neo-institutionalism ought to be the theory of choice when trying to understand firm behaviour in this new and exciting context.

Bookmark and Share