05 Fakultät Informatik, Elektrotechnik und Informationstechnik

Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6

Browse

Search Results

Now showing 1 - 10 of 140

Open Access
Prompt-based personality profiling : filtering social media posts using reinforcement learning
(2024) Hofmann, Jan
Author profiling is the task of inferring characteristics about individuals by analyzing content they share. To date, systems that perform this task automatically, predominantly use supervised machine learning approaches and borrow from advances in the field of natural language understanding. However, while for many language understanding tasks immense progress has been made in recent years (e.g. by using pre-train then fine-tune paradigm), such progress most often does not transfer to automated profiling systems directly. One reason for this is that author profiling is inherently different from typical text inference tasks due to the possibly large amounts of content associated with an author. Therefore, this work proposes a new method for profiling that tries to select the most relevant parts of content shared by an author before inferring a characteristic. Here, instead of relying on ground-truth labels, this work uses the feedback from the zero-shot capabilities of a large language model to learn such a selection model via reinforcement learning, and evaluates this approach for personality profiling in social media. In experiments predicting big five personality traits, this work finds that prediction quality of such a system is comparable yet slightly worse to using all content associated to a profile in a zero-shot setting, while prediction time is reduced significantly due to the limited amount of content used for inferring personality in the proposed method. In addition, this work finds that simply selecting content arbitrarily leads to performance degradation for most traits, and therefore, this work concludes that, to some extent, the proposed method is able to distinguish between relevant and irrelevant content. Further, this work compares the proposed approach to existing supervised approaches and finds that such methods outperform the proposed method substantially. Still, since the ability of the proposed system to distinguish between relevant and irrelevant content of authors is closely tied to the capabilities of large language models, it can be expected that, with advances of such models, prediction quality of the proposed approach will increase in the future.
Open Access
Strukturierte Modellierung von Affekt in Text
(2020) Klinger, Roman; Padó, Sebastian (Prof. Dr.)
Emotionen, Stimmungen und Meinungen sind Affektzustände, welche nicht direkt von einer Person bei anderen Personen beobachtet werden können und somit als „privat“ angesehen werden können. Um diese individuellen Gefühlsregungen und Ansichten dennoch zu erraten, sind wir in der alltäglichen Kommunikation gewohnt, Gesichtsausdrücke, Körperposen, Prosodie, und Redeinhalte zu interpretieren. Das Forschungsgebiet Affective Computing und die spezielleren Felder Emotionsanalyse und Sentimentanalyse entwickeln komputationelle Modelle, mit denen solche Abschätzungen automatisch möglich werden. Diese Habilitationsschrift fällt in den Bereich des Affective Computings und liefert in diesem Feld Beiträge zur Betrachtung und Modellierung von Sentiment und Emotion in textuellen Beschreibungen. Wir behandeln hier unter anderem Literatur, soziale Medien und Produktbeurteilungen. Um angemessene Modelle für die jeweiligen Phänomene zu finden, gehen wir jeweils so vor, dass wir ein Korpus als Basis nutzen oder erstellen und damit bereits Hypothesen über die Formulierung des Modells treffen. Diese Hypothesen können dann auf verschiedenen Wegen untersucht werden, erstens, durch eine Analyse der Übereinstimmung der Annotatorinnen, zweitens, durch eine Adjudikation der Annotatorinnen gefolgt von einer komputationellen Modellierung, und drittens, durch eine qualitative Analyse der problematischen Fälle. Wir diskutieren hier Sentiment und Emotion zunächst als Klassifikationsproblem. Für einige Fragestellungen ist dies allerdings nicht ausreichend, so dass wir strukturierte Modelle vorschlagen, welche auch Aspekte und Ursachen des jeweiligen Gefühls beziehungsweise der Meinung extrahieren. In Fällen der Emotion extrahieren wir zusätzlich Nennungen des Fühlenden. In einem weiteren Schritt werden die Verfahren so erweitert, dass sie auch auf Sprachen angewendet werden können, welche nicht über ausreichende annotierte Ressourcen verfügen. Die Beiträge der Habilitationsarbeit sind also verschiedene Ressourcen, für deren Erstellung auch zugrundeliegende Konzeptionsarbeit notwendig war. Wir tragen deutsche und englische Korpora für aspektbasierte Sentimentanalyse, Emotionsklassifikation und strukturierte Emotionsanalyse bei. Des Weiteren schlagen wir Modelle für die automatische Erkennung und Repräsentation von Sentiment, Emotion und verwandten Konzepten vor. Diese zeigen entweder bessere Ergebnisse, als bisherige Verfahren oder modellieren Phänomene erstmalig. Letzteres gilt insbesondere bei solchen Methoden, welche auf durch uns erstellte Korpora ermöglicht wurden. In den verschiedenen Ansätzen werden wiederkehrend Konzepte gemeinsam modelliert, sei es auf der Repräsentations- oder der Inferenzebene. Solche Verfahren, welche Entscheidungen im Kontext treffen, zeigen in unserer Arbeit durchgängig bessere Ergebnisse, als solche, welche Phänomene getrennt betrachten. Dies gilt sowohl für den Einsatz künstlicher neuronaler Netze, als auch für die Verwendung probabilistischer graphischer Modelle.
Open Access
Tailoring language for user backgrounds : human and LLM evaluations of paraphrased texts
(2025) Carlon, Francesca
In an increasingly interdisciplinary world, the ability to tailor technical and field-specific language to diverse audiences has become essential. Both academic research and industry benefit from the knowledge and expertise of individuals from varied technical backgrounds, supporting progress in research and product development. Therefore, it has become essential to make domain-specific concepts accessible and understandable to different audiences. This study investigates how Large Language Models (LLMs) can generate effective paraphrases by tailoring language to specific target audiences. It also explores which linguistic elements enhance conceptual understanding and whether LLMs can produce high-quality synthetic data for evaluating text quality. This research specifically focuses on tailoring language from Linguistics, Computer Science, and Computational Linguistics texts for audiences with diverse academic backgrounds, using a variety of LLMs. Methods employed include Prompt Engineering, linguistic metrics, and evaluations conducted by both human judges and LLM-as-a-judge approaches. The experiments focus on assessing prompt quality, examining the relationship between human evaluations and linguistic features, and analyzing the correlation between human judgments and LLM-as-a-judge assessments. The results demonstrate that combining Role Prompting with Chain-of-Thought (CoT) prompting effectively produces high-quality paraphrased texts. Additionally, syntactic and lexical features play a central role in enhancing text accessibility and users’ understanding of unfamiliar concepts. Finally, while the correlation between LLM-as-a-judge evaluations and human judgments is promising, it varies across different user groups. Therefore, this research demonstrates the potential of Prompt Engineering techniques for paraphrasing and tailoring language to users with different backgrounds, explores the linguistic features that enhance text comprehension, and examines the extent to which LLM-as-a-judge correlates with human preferences and can support the automatic retrieval of high-quality data.
Open Access
Question answering on knowledge bases : A comparative study
(2021) Kanjur, Vishnudatha
Question Answering intends to automatically extract accurate and relevant information as the answer to a particular question. A large amount of data from the Web is stored as Knowledge bases in a structured way. Question answering on Knowledge bases is a research field that involves multiple branches of computer science like natural language processing, information retrieval and artificial intelligence. Knowledge Base Question Answering (KBQA) research involves various challenges to be solved in multiple aspects. This thesis aimed to compare several state-of-the-art methods for single relation KBQA. The widely used standard single relation dataset, SimpleQuestions dataset was used in the study against Freebase Knowledge Base (KB). A comprehensive analysis of the underlying models and their architecture was performed. Furthermore, to identify the drawbacks and possible enhancements, several approaches for evaluating the models were explored. The results show how the models were performed and the suitability of considering them for solving real-world problems in question answering.
Open Access
Emotion classification based on the emotion component model
(2020) Heindl, Amelie
The term emotion is, despite its frequent use, still mysterious to researchers. This poses difficulties on the task of automatic emotion detection in text. At the same time, applications for emotion classifiers increase steadily in today's digital society where humans are constantly interacting with machines. Hence, the need for improvement of current state-of-the-art emotion classifiers arises. The Swiss psychologist Klaus Scherer published an emotion model according to which an emotion is composed of changes in the five components cognitive appraisal, physiological symptoms, action tendencies, motor expressions, and subjective feelings. This model, which he calls CPM gained reputation in psychology and philosophy, but has so far not been used for NLP tasks. With this work, we investigate, whether it is possible to automatically detect the CPM components in social media posts and, whether information on those components can aid the detection of emotions. We create a text corpus consisting of 2100 Twitter posts, that has every instance labeled with exactly one emotion and a binary label for each CPM component. With a Maximum Entropy classifier we manage to detect CPM components with an average F1-score of 0.56 and average accuracy of 0.82 on this corpus. Furthermore, we compare baseline versions of one Maximum Entropy and one CNN emotion classifier to extensions of those classifiers with the CPM annotations and predictions as additional features. We find slight performance increases of up to 0.03 for the F1-score for emotion detection upon incorporation of CPM information.
Open Access
Effects of paraphrasing and demographic metadata on NLI classification performance
(2023) Marx Larre, Miguel
Native language identification (NLI) refers to the task of automatically deducing the native language (L1) of a document's author, when the document is written in a second language (L2). Documents stem from different sources, but recently more documents are altered before publication through paraphrasing methods. This alteration changes the content, grammar, and style of the document, which inherently obfuscates the L1 of the author. In addition, the demographic metadata of the author, such as age and gender, may influence the performance with which an author's L1 may be detected. In this thesis, two corpora which provide necessary demographic metadata, the International Corpus of Learner English (ICLE) and the \textsc{Trustpilot} corpus, are used to analyze the impact of paraphrasing and demographic factors in the context of NLI tasks. To analyze the effect of paraphrasing on a document, new versions of both corpora are created, which contain paraphrased versions of the documents contained. The effect is inspected using two state-of-the-art NLI systems to perform the task, while the results were analyzed using a regression analysis in combination with dominance analysis (DA). Paraphrasing was found to have a substantial influence in performance of NLI tasks, regardless of corpus, classifier, or paraphrasing method. The usual influence of demographic factors on NLI tasks could not be confirmed in this thesis. Regression analysis and DA allowed for a more profound analysis of the results, which allowed for findings regarding the influence of specific L1s on performance of NLI tasks.
Open Access
Prompt-based continual learning for visual question answering
(2024) Ostertag, Magnus
In an ever-evolving world, Continual Learning (CL) strives to enable a costly trained model to learn new tasks without forgetting previously acquired knowledge. This work critically examines current CL benchmarks for Visual Question Answering (VQA), identifying significant shortcomings in the construction introducing bias. To address these issues, we propose a new CL-VQA benchmark based on GQA, designed to be incremental in both the language and the visual modality. Combined with learning it in one modality only, it can offer rich new diagnostics for a model. Additionally, we extend DualPrompt, a prompt-based CL method, DualPrompt, to the multi-modal domain. Using Dark Experience Replay as a baseline, we evaluate the performance against the new benchmark.
Open Access
Evaluating methods of improving the distribution of data across users in a corpus of tweets
(2023) Milovanovic, Milan
Corpora created from social network data often serve as the data source for tasks in natural language processing. Compared to other, more standardized corpora, social media corpora have idiosyncratic properties due to the fact that they consist of user-generated comments. These are, for example, the unbalanced distribution of the respective comments, a generally lower linguistic quality, and an inherently unstructured and noisy nature. Using a Twitter-generated corpus, I will investigate to what extent the unbalanced distribution of the data has an influence on two downstream tasks, relying on word embeddings. Word embeddings are a ubiquitous and frequently used concept in the field of natural language processing. The most common models are often the means to obtain semantic information about words and their usage by representing the words in an abstract word vector space. The basic idea is that semantically similar words in the mapped vector space have similar vectors. In doing so, these vectors serve as input for standard downstream tasks such as word similarity and semantic change detection. One of the most common models in current research is the use of word2vec, and more specifically, the Skip-gram architecture of this model. The Skip-gram architecture attempts to predict the surrounding words based on the current word. The data on which this architecture is trained greatly influences the resulting word vectors. In the context of this work, however, no significant improvement in the results to a fully preprocessed corpus could be found when filtering methods, widely used in the literature, without specific motivation, are used to select a subset of data according to defined criteria, neither for word similarity nor for semantic change detection. However, comparable results could be achieved with some filters, although the resulting models were trained using significantly fewer tokens as input.
Open Access
Automatic term extraction for conventional and extended term definitions across domains
(2020) Hätty, Anna; Schulte im Walde, Sabine (apl. Prof. Dr.)
A terminology is the entirety of concepts which constitute the vocabulary of a domain or subject field. Automatically identifying various linguistic forms of terms in domain-specific corpora is an important basis for further natural language processing tasks, such as ontology creation or, in general, domain knowledge acquisition. As a short overview for terms and domains, expressions like 'hammer', 'jigsaw', 'cordless screwdriver' or 'to drill' can be considered as terms in the domain of DIY (’do-it-yourself’); 'beaten egg whites' or 'electric blender' as terms in the domain of cooking. These examples cover different linguistic forms: simple terms like 'hammer' and complex terms like 'beaten egg whites', which consist of several simple words. However, although these words might seem to be obvious examples of terms, in many cases the decision to distinguish a term from a ‘non-term’ is not straightforward. There is no common, established way to define terms, but there are multiple terminology theories and diverse approaches to conduct human annotation studies. In addition, terms can be perceived to be more or less terminological, and the hard distinction between term and ‘non-term’ can be unsatisfying. Beyond term definition, when it comes to the automatic extraction of terms, there are further challenges, considering that complex terms as well as simple terms need to be automatically identified by an extraction system. The extraction of complex terms can profit from exploiting information about their constituents because complex terms might be infrequent as a whole. Simple terms might be more frequent, but they are especially prone to ambiguity. If a system considers an assumed term occurrence in text, which actually carries a different meaning, this can lead to wrong term extraction results. Thus, term complexity and ambiguity are major challenges for automatic term extraction. The present work describes novel theoretical and computational models for the considered aspects. It can be grouped into three broad categories: term definition studies, conventional automatic term extraction models, and extended automatic term extraction models that are based on fine-grained term frameworks. Term complexity and ambiguity are special foci here. In this thesis, we report on insights and improvements on these theoretical and computational models for terminology: We find that terms are concepts that can intuitively be derstood by lay people. We test more fine-grained term characterization frameworks that go beyond the conventional term/‘non-term’-distinction. We are the first to describe and model term ambiguity as gradual meaning variation between general and domain-specific language, and use the resulting representations to prevent errors typically made by term extraction systems resulting from ambiguity. We develop computational models that exploit the influence of term constituents on the prediction of complex terms. We especially tackle German closed compound terms, which are a frequent complex term type in German. Finally, we find that we can use similar strategies for modeling term complexity and ambiguity computationally for conventional and extended term extraction.
Open Access
Kategorisierung der Zustandsveränderungen bei CoS-Verben auf Basis von Bild- und/oder Textdaten
(2023) Godbersen, Jule
Sowohl textliche als auch visuelle Informationen können für das Verständnis einer Aktion relevant sein. In dieser Bachelorarbeit werden Aktionen betrachtet, die zu einer Veränderung im Zustand des beteiligten Objekts führen. Ziel dabei ist die Beantwortung der Forschungsfrage, welchen Beitrag die Modalitäten bei der Vorhersage von solchen Zustandsveränderungen haben. Die Vorhersage erfolgt mithilfe von Kategorien wie beispielsweise Farbe, Größe und Quantität. Ein wesentlicher Bestandteil dieser Bachelorarbeit ist die Erstellung eines Datensatzes, der Beispielvorkommen von Aktionen mit Zustandsveränderungen enthält. Eine weitere Aufgabe besteht darin, einiger dieser Datenpunkte mit Kategorien von Zustandsveränderungen annotieren zu lassen. Darüber hinaus wird ausgehend von einem visiolinguistischen Modell eine Ablationsstudie durchgeführt. Diese erlaubt mithilfe verschiedener Klassifikatoren, den Einfluss der verschiedenen Modalitäten auf die Leistungsfähigkeit eines Modells im Hinblick auf die Vorhersage von Zustandsveränderungen zu testen. Diese Bachelorarbeit veranschaulicht unter anderem Schwierigkeiten im Rahmen der Annotationen. Die Leistungsfähigkeit bezüglich der Vorhersage von Kategorien, gemessen mit der Akkuratheit, ist bei den Klassifikatoren ähnlich hoch wie bei einem Baseline Modell. Die verschiedenen Klassifikatoren treffen Vorhersagen mit ähnlicher Akkuratheit, sodass die Forschungsfrage mit den Ergebnissen dieser Bachelorarbeit nicht hinreichend beantwortet werden kann. Die Hypothese, dass die Kombination aus textlicher und visueller Modalität komplementäre Informationen liefert und dementsprechend die Kombination beider Modalitäten relevant ist, wird durch die Ergebnisse nicht bestätigt. Ergänzend wird durch diese Bachelorarbeit gezeigt, dass die trainierten Klassifikatoren es ermöglichen, in gewissem Maße auf ungesehene Datenpunkte, ungesehene Verben und ungesehene Domänen zu generalisieren.

05 Fakultät Informatik, Elektrotechnik und Informationstechnik

Browse

Filters

Settings

Sort By

Results per page

Search Results