Text Classification for Organizational Researchers: A Tutorial

baby-84626_1920[We’re pleased to welcome author Dr. Stefan Mol of the University of Amsterdam. Dr. Mol recently published an article in Organizational Research Methods entitled “Text Classification for Organizational Researchers: A Tutorial,” which is currently free to read for a limited time. Below, Dr. Mol reflects on the inspiration for conducting this research:]

07ORM13_Covers.inddWhat motivated you to pursue this research?
Machine Learning assisted text analysis is still uncommon in organizational research, although its use holds promise. Most manual text analysis procedures conducted by researchers in this field are about the assignment of text to categories such as in thematic and template analyses. However, manual classification of text becomes laborious and time consuming (and sometimes subject to reliability issues) when one needs to do this for a sizeable amount (hundreds of thousands or millions) of pieces of text. An alternative is to use automatic text classification systems that can be constructed by researchers, which allow them to speed up the process of labeling or coding large sets of textual data. The design and building of text classifiers could be of use for various areas of organizational research. Our aim was to illustrate how this could be done and provide a tutorial. We used the example of building a text classifier to automatically sort job type information contained in job vacancies. The importance of validating the results of text classification was demonstrated through data triangulation, using expert input. We believe that the use of this procedure among organizational researchers can improve reliability and efficiency in analysis that involves classification.
What has been the most challenging aspect of conducting your research? Were there any surprising findings?
Building classifiers involves several rounds of training, testing, and validation before they can be deployed in practice and the most challenging aspect is training the classifier and choosing the parameters in such a way that the results are valid from the standpoint of application. The classifier we built for the job analysis task was able to recover job task sentences with high precision as assessed by an expert in the field, although the classifier was initially trained with minimum expert input. Our results thus suggest that job vacancies are a reliable alternative source of job information that can augment existing approaches to job analysis. More generally, we believe this also suggests that wider use of text classification holds promise for organizational research in a broader sense.
What did not make it into your published manuscript that you would like to share with us?
One class of techniques that are now increasingly applied in the area of text classification are word embeddings. Word embeddings map each word to vectors of real numbers. The similarities among word vectors can be used to quantify and categorize the meaning of words in specific contexts. We initially planned to include a short discussion about this but we decided not to because these techniques warrant more in depth discussion which go beyond the scope of our current article. However, organizational researchers interested in recovering context specific meaning of words may benefit from the specific approach taken with word embeddings and we recommend them to get to know these techniques as well.

Stay up-to-date with the latest research from Organizational Research Methods and sign up for email alerts today through the homepage!

 

 

 

Want to Spread Your Message on Facebook?

Dr. Linchi Kwok, Syracuse University

Editor’s note: We are pleased to welcome Linchi Kwok, assistant professor of Hospitality Management in the David B. Falk College of Sport and Human Dynamics at Syracuse University, whose research interests include social media and its business implications, organizational behavior, and service operations. Dr. Kwok and Dr. Bei Yu, also of Syracuse University, published “Spreading Social Media Messages on Facebook: An Analysis of Restaurant Business-to-Consumer Communications” on September 24 in Cornell Hospitality Quarterly.

***

Want to Spread Your Social Media Messages on Facebook?
This Study May Help

Being a phenomenologist and a practitioner of social media, I see Facebook as one of the most important means for B2C (business-to-consumer) communications. When a Facebook user likes, posts comments, or shares content with their Facebook credentials, an update will appear on this person’s wall, helping companies rapidly spread information. Thus, companies must pay close attention to Facebook users’ reactions to the messages they send on Facebook. Facebook users’ endorsement of a message can be very important in indicating the effectiveness of a company’s social media strategy.

Dr. Yu and I adopted the text mining techniques to identify the type(s) of Facebook that are endorsed (and thus propagated) by Facebook users. We analyzed 982 Facebook messages initiated by 10 restaurant chains and two independent operators, of which were among the top restaurants in terms of sales volume and number of Facebook fans. We found the following results: the “more popular” messages, which receive more “Likes” and comments, contain keywords about the restaurant (e.g., menu descriptions); the “less popular” messages seem to involve with sales and marketing. Dividing the messages into four media types (i.e., status, link, video, and photo), photo and status receive more “Likes” and comments. To dig further, we coded the messages into two message types, namely sales/marketing and conversational messages, which do not directly sell or promote the restaurants. As compared to sales and marketing messages, conversational messages receive more “Likes” and comments even though they only account for one third of the messages in this study. There is also a cross-effect of media type and message type on the number of comments a message received.

Based on the research findings, we outlined several detailed practical tactics in this paper to help companies improve their use of Facebook. Theoretically, the findings of this study provide ground work for developing a defined typology of Facebook messages and an automatic text classifier with the machine learning techniques.

***
Click here to read the article in the OnlineFirst section of Cornell Hospitality Quarterly. Follow this link to learn more about the journal and this one to receive e-alerts about newly published articles that provide timely and actionable prescription for hospitality management practice and research.