Psychological Science Seeking Preregistered Replications

 [The following post is re-blogged from Method Space. Click here to view the original article.]

keys-2114366_640_opt

Making a statement in the ongoing “replication” or “reproducibility crisis,” the journal Psychological Science will now accept a special class of research–based papers that report on attempts to re-create experiments that had influential findings and that were first published in Psychological Science.
This new category of “preregistered direct replications,” or PDRs, aims “to create conditions that experts agree test the same hypotheses in essentially the same way as the original study,” explained D. Stephen Lindsay, a cognitive psychologist at the University of Victoria and editor of Psychological Science.

That psychology in particular is in need of a replication tonic has been widely accepted. “To me,” Jim Grange, a senior lecturer in psychology at Keele University, wrote last year, “it is clear that there is a reproducibility crisis in psychological science, and across all sciences. Murmurings of low reproducibility began in 2011 – the ‘year of horrors’ for psychology – with a high profile fraud case. But since then, The Open Science Collaboration has published the findings of a large-scale effort to closely replicate 100 studies in psychology. Only 36 percent of them could be replicated.”

Under the PDR description, the ‘direct’ part of the replication rubric refers to sticking to the original methodology as closely as possible. “It is impossible to conduct a study twice in exactly the same way,” Lindsay said in a letter announcing the PDRs, “and doing so would not always be desirable (e.g., often it would be better to test a larger number of subjects than in the original study; if the original study included some problematic items or unclear instructions, it is probably best to correct those shortcomings; if the original and replication studies were conducted in different cultural contexts then it may be appropriate to change surface-level aspects of the procedure).”

It is expected, Lindsay said, that the author of the target piece will be invited to provide a review of the replication, along with at least two independent experts. And in keeping with the ‘preregistered’ aspect of the PDR, researchers are asked to submit proposals for review before data collection begins. “A virtue of pre-data-collection submissions is that decisions about scientific merit are independent of how the results come out,” he said. “Another advantage is that reviewers and the editor have an opportunity to improve a study before it is conducted. We may therefore at some point in the future require that PDRs be submitted as proposals prior to data collection.  But for now we will also consider PDRs that have already been conducted if they were preregistered.” And in keeping with an earlier announcement also focused on transparency, Lindsay said would-be authors are asked to provide the data they use in their published analysis to reviewers.

Lindsay made clear that while the replication needed to leverage an article that originally appeared in Psychological Science, merely having appeared there was not sufficient for a PDR. “The primary criterion is general theoretical significance,” he emphasized.

Still, he said, the journal – which takes its lumps from some observers like Andrew Gelman – has a responsibility to not turn its back on what it’s published. “One of the motivations for adding PDRs is the belief that a journal is responsible for the works it publishes (as per Sanjay Srivastava’s ‘Pottery Barn rule’ blog post). That said,” he continued, “our goal is not to publish ‘gotchas!’ Rather, inviting PDRs is one of many steps taken to increase the extent to which this journal contributes to efforts to make psychology a more cumulative science.”

Regardless with whether a replication confirms or question the original findings, the outcomes from the PDRs will be “valuable and informative.” He cited a 2009 column by then Association for Psychological Science President Walter Mischel which noted “replications sometimes yield more nuanced results that spark new hypotheses and contribute to the elaboration of psychological theories.”

The PDRs will launch with a replication of a 2008 an fMRI study Psychological Science published in 2008, said journal editor Steve Lindsay. That original study, conducted by William A. Cunningham, Jay J. Van Bavel and Ingrid R. Johnsen, detailed evidence that processing goals can modulate activity in the amygdala, a part of the brain that influences memory, emotions and decision-making. The new article explains how the University of Denver’s Kateri McRae and the University of California, Los Angeles’ Daniel Lumian conducted a high-powered replication of that experiment — in consultation with Cunningham and Van Bavel – and in the process replicated the original’s central finding.

“This article,” said Lindsay, “is a fine model of a preregistered direct replication,” in part because of its greater statistical power and multiple converging analyses compared to the original.

Lindsay said PDRs are not the same as the registered replication reports that have appeared in the sister journal, Perspectives on Psychological Science, and which along with other multi-lab empirical papers will appear in the new journal, Advances in Methods and Practices in Psychological Science.

The Evolution and Prospects of Service-Dominant Logic Research

[We’re pleased to welcome author Dr. Ralf Wilden of Newcastle Business School, University of Newcastle, Australia. Dr. Wilden recently published an article in the Journal of Service Research entitled “The Evolution and Prospects of Service-dominant Logic Research: An Investigation of Past, Present, and Future Research,” which is currently free to read for a limited time.” Below, Dr. Wilden reflects on the inspiration for conducting this research:]

02JSR13_Covers.inddService innovation is a driving force of economic growth in developed economies. Large corporations, such as BMW and IBM, increasingly define their business as service centric. For example, the BMW Group has moved away from defining their value proposition being focused on cars and motorcycles to positioning themselves as a mobility provider, thus moving away from a product-centered to service-centered narrative. The ‘servitization’ of traditional business models converges with a growing academic discourse around the emergence and evolution of the so-called ‘service-dominant logic’. Ongoing studies in this area explore the value of service in dynamic exchange systems and how managers are responding to or guided by ideas that 1) service forms the basis of all economic exchange, 2) value is always co-created between relevant actors, and 3) so-called operant resources are central to value co-creation.

In a recent study in the Journal of Service Research, an international team of researchers studied existing research to uncover core concepts and thematic shifts in the development of new knowledge in this field. More specifically, they studied how service-dominant logic advances the understanding of how value is created and service is innovated in dynamic service ecosystems. Based on a citation analyses and text mining of more than 300 key articles, the authors identify how service-dominant logic bridges traditional service research (e.g., regarding satisfaction, quality and customer experiences) with strategic and systems views. However, looking at the evolution of service-dominant logic research over time, it appears focus on strategic research has waned. Thus, the authors argue future studies should draw on several specific research areas to develop frameworks to aid managers in strategically thinking about service design and innovation.

The results from this study verify service-dominant logic is highly influential in areas such as customer engagement and value cocreation. An underlying shift towards social and systemic perspectives is also evident. However, many valuable insights emerging from the wealth of relevant studies have not yet impacted research regarding managerial decision-making and strategy development on a large scale. Furthermore, the authors identify the need to develop a stronger understanding of the way service-dominant logic can be used to inform how managerial actions and social and cultural practices influence and are influenced by a wider service ecosystem. For example, Ralf Wilden says “the way organizations engage in innovation-related activities has changed from a firm-centric model to a model that stresses the importance of knowledge in-flows and out-flows across organizational boundaries.” He adds, “despite the commonly accepted importance of services in value creation activities our knowledge about the role of open innovation in service ecosystems is limited.” The authors further stress that service thinking has benefited from interdisciplinary research in the past. Moving forward, combining service-dominant research with organizational strategy insights in the area of open innovation, dynamic capabilities and microfoundations, together with social, cultural and systems theories, can lead to developing new knowledge regarding service and drive continual improvement in service design and innovation.

Visit the journal homepage to sign up for email alerts!

The Mind-Set of Editors and Reviewers

Get the latest insight on what editors are looking for in your submitted manuscript! SAGE Publishing is proud to feature the latest editorial from Family Business Review, entitled, “The Mind-Set of Editors and Reviewers.” This editorial is co-authored by James J. Chrisman, Pramodita Sharma and Jess Chua, and is currently free to read for a limited time.

Below, please find an excerpt from the editorial, shedding light on the necessary steps an author must face when preparing a manuscript that stands out:

The formula for getting a manuscript published seems deceptively simple, with an emphasis on deceptively. For family business research, the four-step process starts with authors coming up with interesting research questions, that when addressed, will change scholarly understanding of the motivation, behavior, or performance of family firms. As elaborated in the editorial by Salvato and Aldrich (2012), while there are many sources of inspiration for generating interesting research questions, in professional fields like family business studies, researchers with closer linkages to practice and/or prior literature are better positioned to identify questions that lead to usable knowledge that is not only published but also well-read and cited (cf. Lindblom & Cohen, 1979). Objectives such as simply “getting published” may be more dominant in earlier career stages. Over time, however, most scholars hope to make a difference in the mind-sets of other researchers and ultimately practitioners (Vermeulen, 2007; Zahra & Sharma, 2004). But, this does not always happen.

Click here to read the full article. Don’t forget to sign up to receive email alerts so you never miss the latest research from Family Business Review!

Call for Proposals! Small Group Research: 2018 Review Issue

SGR_call2018.jpg

Small Group Research is currently accepting proposals regarding the 2018 Review Issue. Please view the full details about the submission process here, or by clicking on the image above.

Manuscripts are due:  May 30, 2017

Submissions should be made to SGR’s Manuscript Central website: http://mc.manuscriptcentral.com/sgr, where authors will be required to set up an account.

Small Group Research (SGR), peer-reviewed and published bi-monthly, is an international and interdisciplinary journal presenting research, theoretical advancements, and empirically supported applications with respect to all types of small groups. SGR, a leader in the field, addresses and connects three vital areas of study: the psychology of small groups, communication within small groups,and organizational behavior of small groups. This journal is a member of the Committee on Pubication Ethics (COPE).

Don’t forget to sign up for email notifications about the latest research through the journal homepage.

Benefits and Costs of Covert Research: An Analysis

[We’re pleased to welcome author Thomas Roulet of King’s College, UK.  Roulet recently published an article in Organizational Research Methods entitled Reconsidering the Value of Covert Research: The Role of Ambiguous Consent in Participant Observation, co-authored by Michael J. Gill, Sebastien Stenger, and David James Gill. From Roulet:]

What inspired you to be interested in this topic? We were inspired by recent ethnographic work relying heavily on covert observation – for example the recent work by Alice Goffman on low income urban areas, or the paper byORM_72ppiRGB_powerpoint.jpg Ethan Bernstein on the pitfalls of transparency in a Chinese factory.Alice Goffman’s work was attacked for the ethical challenges associated with the work of ethnographer.

So we went back to the literature and looked at research in various fields that relied on covert observation – the observation of a field of enquiry by a researcher that does not reveal his or her true identity and motives. This methodological approach has progressively fallen into abeyance because of the ethical issues associated with it- in particular the fact that covert observation implies not getting consent from the people observed by the researcher.

Were there findings that were surprising to you? Our review of covert research reveals that:
– all observations have issues with consent to different extent. It is obtain the full consent of all subjects. We put forward a two dimensions
– covert research can be ethically justified when tackling taboo topics, or trying to uncover misbehaviors.
– there is a wide range of ways and practices that can be used to minimize ethical concerns and limit the harm to subjects.

How do you see this study influencing future research? We hope that the ethical guidelines of some associations can evolve to offer more room for covert or semi covert research, and acknowledge the difficulties of obtaining full consent. We also think that ethical boards in universities may be willing to offer a more flexible perspective on covert research.

Finally our work is a call to researchers to consider covert observational approaches… with care!

Don’t forget to sign up for email alerts through the ORM homepage. 

How Surveys Provide Integrated Communication Skills

“Excuse me, can you spare a  a few minutes? We’re conducting a survey and would greatly appreciate your responses.” You’ve most likely heard these two sentences presented to you as you’re walking briskly down a crowded street. The Internet is also a crowded street full of news, but we hope you can spare a few minutes to read about the latest research from Business and Professional Communication Quarterly.

Author Anne Witte of EDHEC Business School, France, recently published a paper in BCQ entitled “Tackling the survey: A learning-by-induction design,”where she outlines the different learning outcomes that surveys afford. Below, Witte describes her inspiration for the study:]

  • What inspired you to be interested in this topic?

Our world is filled with surveys, yet surveys are often a negl4453697565_dcacd29f08_z.jpgected area in business training and often taught as a kind of mechanical application task which has more to do with software than with thinking.  As qualitative and quantitative data are the basis for business and organizations today, I wanted to train students more in the “art” rather than the “science” of the survey.

  • Were there findings that were surprising to you?

Students are often overconfident in their ability to do a survey task from A to Z.  When you challenge them with an interesting question to answer through a survey, they discover on their own how difficult it really is to obtain quality data that can be used to make decisions.

  • How do you see this study influencing future research and/or practice?

I love testing new teaching paradigms with advanced business students and especially using interdisciplinary thought experiments that oblige students to draw from previous knowledge and varied skills sets.

Don’t forget to sign up for email alerts on the BCQ homepage. 

Survey photo attributed to Plings (CC).

 

Communities as Nested Servicescapes

jsrrr.JPG[We’re pleased to welcome Xiaojing Sheng from the University of Texas at Rio Grande. Sheng co-authored a recently published article in the Journal of Service Research  entitled “Communities as Nested Servicescapes” with Penny Simpson and Judy Siguaw. From Sheng:]

  • What inspired you to be interested in this topic?

From groups of four to sixteen sipping margaritas in local restaurants to dancing at a beach or Mexican fiesta, retired winter migrants are a ubiquitous presence in the Rio Grande Valley of South Texas each winter. These migrating consumers repeatedly come to the area in large numbers each winter to enjoy the warm tropical weather, to participate in the many available activities, and to enjoy each other in their highly social living environment of mobile homes and recreational vehicle communities. These senior citizens also become an inseparable part of the region by routinely going to restaurants, events, shows and stores where they seem to exude a comradery and enjoyment of life not seen by typical residents of any community. For these migrants, winter life in the Valley seems to be a fun-filled, months-long vacation. Through casual observation of the lifestyle of these hundreds of thousands of active retirees, we were driven to understand their experiences as they become immersed in the broader servicescape of the Valley and in the nested servicescapes of their mobile home/recreational vehicle communities in which they reside for extended periods of time.

  • Were there findings that were surprising to you?

The finding that servicescape engagement weakened the positive effect of perceived servicescape satisfaction on loyalty intention is unexpected and surprising. This is probably because high levels of activity engagement become all-consuming, making perceived servicescape satisfaction itself less important in loyalty intention. For example, consumers may be willing to overlook a rundown beach villa if the beach activities are exceptional. On the other hand, lower levels of engagement strengthened the impact of perceived servicescape satisfaction on loyalty intentions, conceivably because consumer attention is less distracted by activity involvement, and therefore, more focused on servicescape factors.

  • How do you see this study influencing future research and/or practice?

An interesting finding from our study is that, when consumers interact with two servicescapes of which one is nested within another, their experiences are shaped by the effects of the individual servicescape, the compounding effects of both servicescapes, and by the transference effects between the two servicescapes. Consequently, marketers need to take a holistic approach to managing servicescapes at all levels to create an overall positive consumer experience. We hope that our research serves as a catalyst for future studies that examine effects of nested servicescapes. Moreover, we hope our work encourages other researchers to investigate less conventional servicescapes, such as regions, towns, and neighborhoods, because there is so much more to be learned about how the places in which we live, work, and play affect and transform our lives.

Don’t forget to sign up for email alerts through JSR’s homepage so you never miss a newly published article.