Methods: Machine learning, large language models, causal inference
Cheng, Mengjie, Elie Ofek, Hema Yoganarasimhan. Balancing Engagement and Polarization: Multi-Objective Alignment of News Content Using LLMs. [Paper]
Marketing Science
Revise & resubmit.
We study how media firms can use LLMs to generate news content that aligns with multiple objectives—making content more engaging while maintaining a preferred level of polarization/slant consistent with the firm’s editorial policy. Using news articles from The New York Times, we first show that more-engaging human-written content tends to be more polarizing. Further, naively employing LLMs (with prompts or standard Direct Preference Optimization approaches) to generate more engaging content can also increase polarization. This has an important managerial and policy implication: using LLMs without building in controls for limiting slant can exacerbate news media polarization. We present a constructive solution to this problem based on the Multi-Objective Direct Preference Optimization (MODPO) algorithm, a novel approach that integrates Direct Preference Optimization with multi-objective optimization techniques. We build on open-source LLMs and develop a new language model that simultaneously makes content more engaging while maintaining a preferred editorial stance. Our model achieves this by modifying content characteristics strongly associated with polarization but that have a relatively smaller impact on engagement. Our approach and findings apply to other settings where firms seek to use LLMs for content creation to achieve multiple objectives, e.g., advertising and social media.
Cheng, Mengjie, Shunyuan Zhang. Reputation Burning: Analyzing the Impact of Brand Sponsorship on Social Influencers. [Paper]
Management Science
Pre-published online.
The growth of the influencer marketing industry warrants an empirical examination of the effect of posting sponsored videos on influencers’ reputations. We collected a novel data set of user-generated YouTube videos created by prominent English-speaking influencers in the beauty and style category. We extracted a rich set of theory-driven video features and used DiNardo-Fortin-Lemieux reweighting to construct comparable treatment and control groups matched at the influencer-video level. The empirical analysis of the matched sample revealed a reputation-burning effect; that is, posting a sponsored video, compared with posting an equivalent organic video, cost the influencers 0.19% of their reputation (operationalized as the number of subscribers). The reputation-burning effect was stronger among the influencers with larger audiences. An analysis of likes, comments, and comment texts revealed a larger gap in audience response between sponsored and organic videos among the influencers with larger (versus smaller) audiences. The reputation-burning effect was mitigated when a close fit existed between the sponsored content and the influencer’s “usual” content and when the promoted brand was less well known. Our study empirically tested the assumption of several theoretical works. Moreover, it contributes to the literature on influencer marketing and celebrity endorsements and provides managerial implications for influencers, brands, and social media platforms.
Cheng, Mengjie, Daniel Scott Smith, Xiang Ren, Hancheng Cao, Sanne Smith, Daniel A. McFarland. How New Ideas Diffuse in Science. [Paper]
American Sociological Review 88, 3 (2023): 522–561.
What conditions enable novel intellectual contributions to diffuse and become integrated into later scientific work? Prior work tends to focus on whole cultural products, such as patents and articles, and emphasizes external social factors as important. This article focuses on concepts as reflections of ideas, and we identify the combined influence that social factors and internal intellectual structures have on ideational diffusion. To develop this perspective, we use computational techniques to identify nearly 60,000 new ideas introduced over two decades (1993 to 2016) in the Web of Science and follow their diffusion across 38 million later publications. We find new ideas diffuse more widely when they socially and intellectually resonate. New ideas become core concepts of science when they reach expansive networks of unrelated authors, achieve consistent intellectual usage, are associated with other prominent ideas, and fit with extant research traditions. These ecological conditions play an increasingly decisive role later in an idea’s career, after their relations with the environment are established. This work advances the systematic study of scientific ideas by moving beyond products to focus on the content of ideas themselves and applies a relational perspective that takes seriously the contingency of their success.
Cheng, Mengjie, Elie Ofek, Hema Yoganarasimhan. The Value of Silence: The Effect of UMG’s Licensing Dispute with TikTok on Music Demand. [Paper]
Marketing Science
Under 2nd round review.
Social media platforms like TikTok have transformed how music is discovered, consumed, and monetized. This study examines the implications of the dispute between TikTok and Universal Music Group (UMG), which resulted in UMG excluding its music from TikTok from February to May 2024. UMG claimed that TikTok’s compensation was inadequate, as the presence of its tracks on the platform potentially reduced revenue that could be generated elsewhere. Conversely, TikTok argued that their compensation was appropriate, emphasizing the promotional and discovery benefits for artists. To examine the validity of these conflicting viewpoints, we conduct a difference-in-differences analysis, using tracks from Sony and Warner as a control group. We generally find that removing UMG music from TikTok did not significantly alter the overall demand for UMG tracks on streaming platforms like Spotify and YouTube. However, there is significant heterogeneity across tracks: previously available tracks on TikTok experienced a 2–3% increase in consumption when removed, indicating a substitution effect, predominantly encompassing more popular tracks from well-known artists. Conversely, UMG tracks not previously available on TikTok saw a 1–3% decrease in streams, indicating a complementary effect, mainly encompassing less popular tracks from lesser-known artists. Further analysis suggests that the complementary effect is driven by TikTok’s role in promoting and discovering artists with a partial presence on the platform. An economic impact analysis shows that TikTok significantly undercompensates UMG, aligning with the terms of a new licensing agreement between the parties. This study provides valuable managerial implications for music labels, social media platforms, streaming services, and artists.
Cheng, Mengjie, Tomomichi Amano, Elie Ofek, Yicheng Song. Déjà Vu: How Encountering Similar Content Impacts Social Media Engagement.
Management Science
Revise & resubmit.
We study how engagement with content on social media platforms is affected by a user’s prior post consumption, addressing a gap in the literature that primarily focuses on the static characteristics of focal posts. Using Instagram data from influencers who worked with two major retail beauty brands on their promotional campaigns, we develop an empirical model to assess the impact of content similarity and network structure on user engagement. We find that engagement with a current post is affected by the amount of exposure to past similar content, exhibiting an inverted-U relationship. Additionally, we find that when similar content originates from different influencers, it generates greater engagement relative to comparable content originating from the same influencer. Building on the estimated empirical model, we conduct policy simulations that suggest how influencers can increase engagement with their content by controlling the similarity level of sequential posts. Moreover, the model can provide strategic guidance for brands on the expected performance of a set of influencers for marketing campaigns. Together, our work sheds light on the dynamic nuances of social media engagement and offers practical advice for content creation and influencer selection to improve campaign impact.
Cheng, Mengjie, Max Beichert, and Xitong Li, and Shunyuan Zhang. Predicting Influencer Marketing Effectiveness: A Multi-Task Learning Approach.
Working Paper.
Brands are increasingly partnering with social media influencers to boost products sales. However, predicting which influencers will deliver strong sales outcomes remains challenging, creating a “black box” in influencer marketing. While metrics like follower size help guide influencer selection, many campaigns still fail to meet expectations and even generate no sales at all. These metrics, though readily measurable, miss critical unobserved factors, such as influencers’ own goals and incentives, which also affect the campaign outcomes. While brands focus on sales, influencers are often motivated to prioritize their reputation, which directly impacts future collaborations and income. These differing incentives can lead influencers to make content and engagement choices that don’t always align with brand goals. Complicating matters, brands cannot observe influencer behaviors or control them before or during a campaign. To address this, we propose a multi-task learning (MTL) model that learns shared representations to predict both brand goals (sales) and influencer goals (reputation) simultaneously. Our model not only establishes the effectiveness on several metrics, but also significantly outperforms traditional single task learning models optimized solely for sales. Further analysis suggests that the model is particularly effective when influencers have recently experienced a decline in reputation, possibly due to an increased focus on reputation management. Our work presents a significant advancement in modeling influencer marketing effectiveness and offers brands a valuable approach of optimizing influencer marketing campaigns.
Cheng, Mengjie, Jeremy Yang, Elie Ofek. Fanning the Flames: The Diffusion of Content Recreation on Social Media Platforms.
Draft in preparation.
Cheng, Mengjie, Elie Ofek, Hema Yoganarasimhan. An LLM-Prediction Powered Inference Framework.
Under model development.
* indicates equal contribution.
Cao, Hancheng*, Mengjie Cheng*, Zhepeng Cen*, Daniel A. McFarland, Xiang Ren. Will This Idea Spread Beyond Academia? Understanding Knowledge Transfer of Scientific Concepts across Text Corpora, in Empirical Methods in Natural Language Processing EMNLP (Findings) 2020. [Paper]
Cao, Hancheng*, Zhilong Chen*, Mengjie Cheng*, Shuling Zhao, Tao Wang, Yong Li. You Recommend, I Buy: How and Why People Engage in Instant Messaging Based Social Commerce, in Proceedings of the ACM on Human-Computer Interaction 5 (CSCW 2021), 1–25. [Paper]
Chen Zhilong, Hancheng Cao, Fengli Xu, Mengjie Cheng, Tao Wang, Yong Li. Understanding the Role of Intermediaries in Online Social E-commerce: an Exploratory Study of Beidian, in Proceedings of the ACM on Human-Computer Interaction 4 (CSCW 2020), 1–24.
[Paper]