“Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good” Paper Summary & Analysis
Paper: https://arxiv.org/pdf/1906.06725.pdf
Discussion led by Cora Wu & Winnie Chow, Intelligent Systems subteam
Objectives / Goals of the Paper
The main objective of this paper is to develop the foundations for personalized persuasive conversational agents to change people’s opinions and actions for social good. In the paper, the authors create a classifier to predict persuasive strategies as the groundwork for future research in this area. Although there have already been previously published papers looking into learning better negotiation strategies, Persuasion for Good distinguishes itself through its emphasis on the personalization aspect of the persuasion process. By taking in user background information through sub-questionnaires such as the Big Five personality test alongside conversational data, the paper aims to find relationships between users’ backgrounds and the effectiveness of persuasion strategies.
Paper Contributions
In the paper, the authors collected conversational data of participants trying to convince other participants to donate to a charity using Amazon’s Mechanical Turk. They then annotated the data to indicate the presence of certain persuasion techniques. This allowed for analysis of psychological factors, as well as the persuasion technique being used. Model-wise, they used a hybrid RCNN model which included combinations of different embeddings, ranging from character level to context level embedding. They use this classifier on their annotated data to predict the presence of 10 persuasive strategies.
Finally, the hybrid RCNN model combined the sentence embedding and context embedding with 3 sentence-level features (turn position embedding, semantic embedding, and character embedding) to increase the classifier’s performance by capturing meta-information outside of the sentence and context embeddings.
The paper assessed its results by comparing the classification task performance of the proposed hybrid RCNN with other baseline models. In addition to comparing their results with other models, they also conducted an ablation study to determine the effect of the different features on the model performance. The paper found that the hybrid RCNN outperformed the baseline models, and also found that including the sentence-level features improved the performance of their classifier. In addition, they plotted a confusion matrix with the best performing model and their hybrid RCNN with all features to assess performance on a class level.
Using these methods, Wang et al. accounted for the context of a person’s lived experiences when deciding how to interpret/express certain ideas. After these experiments, for instance, data analysis found that more extroverted individuals were more receptive toward emotional appeals, so potential persuaders would know to attempt an emotional appeal toward a user who displayed the traits that corresponded to a higher degree of extroversion. This level of personalization wasn’t highlighted in older models and can reveal trends in persuasive conversations that can affect stronger reactions in a broader audience.
Paper Limitations
The approach seems somewhat limited in that some arguments may be constructed via some combination of potential approaches. For example, an argument may appeal both to the emotional and logical sensibilities of the persuadee, but the dataset assumedly only allows arguments of exactly one strategy. This is addressed in section 6, but simply assigning the most salient strategy used as the classification of the argument seems limited and prone to noise.
Also, the dataset they compiled only had 300 annotated conversations. Because of the limited amount of data, they couldn’t investigate the discrepancy between persuadees’ claims to donate and whether they actually end up donating. In the original paper, there were only 236 persuadees who agreed to donate, but a large percentage (43%) of those individuals did not end up donating. Additionally, an extra 11% of those individuals reduced the amount they donated compared to what they claimed. Since so many of the participants didn’t actually adhere to their donation claims, we think that this is an important issue to address in the future.
Further Research
Future research can look into how the model can be implemented in practice (how user information should be collected, how to make sure the ethical guidelines mentioned in the paper will be enforced, etc.). The authors also suggest gathering more annotated data and more dialogue context to improve the performance of their classifiers.
The paper’s discovery of evidence of interaction between physiological backgrounds and persuasion strategies has implications in fields outside of natural language processing, and could be extended to psychology as well. The effect of personalized persuasion could be a topic of research in many fields and industries. Moreover, the work of this paper not only gives affirmation to the possibility of, but also lays the groundwork for generative models that can create adaptive persuasive dialogue tailored to a specific end user based on their profile.
However, further research in this area can raise several ethical concerns that should not go unnoticed. Future researchers should carefully consider the ethical implications of using machine learning to sway peoples’ thoughts and opinions.
