What makes a good argument? Investigating subjective factors of argument strength

Abstract

Argument quality assessment is a field of computational argument mining, in which the quality or strength of persuasive texts is rated automatically. The notions of what makes a good argument are manifold. Historically, argument quality pertained mostly to objective markers like clarity, logical soundness or coherence. As the field shifts to address subjectivity and persuasion, the definition of argument strength also broadens to include persuasiveness and the subjective complexities this shift brings with it. While many small studies on subjective features of arguments exist, there are no large-scale analyses of the relation between these features and argument strength. To address this gap, I model the influence of three subjective features on argument quality data from differently focused domains. My contribution is twofold: first, I conduct a regression analysis on argument strength with the features of storytelling, emotions and hedging, which argument research either approached onesidedly or only recently. Secondly, as there are no datasets available with annotations for all four dimensions, I compare different methods for automatically annotating argument data with labels for storytelling, emotions and hedging. My analysis shows a link between the features and argument strength as well as systematic differences between the two argument corpora. In evaluating different automatic annotation methods, I find advantages of modified training setups but also see some limitations in how far automatic methods reach for complex tasks like cross-domain emotion classification.