05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
118 results
Search Results
Item Open Access Prompt-based personality profiling : filtering social media posts using reinforcement learning(2024) Hofmann, JanAuthor profiling is the task of inferring characteristics about individuals by analyzing content they share. To date, systems that perform this task automatically, predominantly use supervised machine learning approaches and borrow from advances in the field of natural language understanding. However, while for many language understanding tasks immense progress has been made in recent years (e.g. by using pre-train then fine-tune paradigm), such progress most often does not transfer to automated profiling systems directly. One reason for this is that author profiling is inherently different from typical text inference tasks due to the possibly large amounts of content associated with an author. Therefore, this work proposes a new method for profiling that tries to select the most relevant parts of content shared by an author before inferring a characteristic. Here, instead of relying on ground-truth labels, this work uses the feedback from the zero-shot capabilities of a large language model to learn such a selection model via reinforcement learning, and evaluates this approach for personality profiling in social media. In experiments predicting big five personality traits, this work finds that prediction quality of such a system is comparable yet slightly worse to using all content associated to a profile in a zero-shot setting, while prediction time is reduced significantly due to the limited amount of content used for inferring personality in the proposed method. In addition, this work finds that simply selecting content arbitrarily leads to performance degradation for most traits, and therefore, this work concludes that, to some extent, the proposed method is able to distinguish between relevant and irrelevant content. Further, this work compares the proposed approach to existing supervised approaches and finds that such methods outperform the proposed method substantially. Still, since the ability of the proposed system to distinguish between relevant and irrelevant content of authors is closely tied to the capabilities of large language models, it can be expected that, with advances of such models, prediction quality of the proposed approach will increase in the future.Item Open Access Tailoring language for user backgrounds : human and LLM evaluations of paraphrased texts(2025) Carlon, FrancescaIn an increasingly interdisciplinary world, the ability to tailor technical and field-specific language to diverse audiences has become essential. Both academic research and industry benefit from the knowledge and expertise of individuals from varied technical backgrounds, supporting progress in research and product development. Therefore, it has become essential to make domain-specific concepts accessible and understandable to different audiences. This study investigates how Large Language Models (LLMs) can generate effective paraphrases by tailoring language to specific target audiences. It also explores which linguistic elements enhance conceptual understanding and whether LLMs can produce high-quality synthetic data for evaluating text quality. This research specifically focuses on tailoring language from Linguistics, Computer Science, and Computational Linguistics texts for audiences with diverse academic backgrounds, using a variety of LLMs. Methods employed include Prompt Engineering, linguistic metrics, and evaluations conducted by both human judges and LLM-as-a-judge approaches. The experiments focus on assessing prompt quality, examining the relationship between human evaluations and linguistic features, and analyzing the correlation between human judgments and LLM-as-a-judge assessments. The results demonstrate that combining Role Prompting with Chain-of-Thought (CoT) prompting effectively produces high-quality paraphrased texts. Additionally, syntactic and lexical features play a central role in enhancing text accessibility and users’ understanding of unfamiliar concepts. Finally, while the correlation between LLM-as-a-judge evaluations and human judgments is promising, it varies across different user groups. Therefore, this research demonstrates the potential of Prompt Engineering techniques for paraphrasing and tailoring language to users with different backgrounds, explores the linguistic features that enhance text comprehension, and examines the extent to which LLM-as-a-judge correlates with human preferences and can support the automatic retrieval of high-quality data.Item Open Access Effects of paraphrasing and demographic metadata on NLI classification performance(2023) Marx Larre, MiguelNative language identification (NLI) refers to the task of automatically deducing the native language (L1) of a document's author, when the document is written in a second language (L2). Documents stem from different sources, but recently more documents are altered before publication through paraphrasing methods. This alteration changes the content, grammar, and style of the document, which inherently obfuscates the L1 of the author. In addition, the demographic metadata of the author, such as age and gender, may influence the performance with which an author's L1 may be detected. In this thesis, two corpora which provide necessary demographic metadata, the International Corpus of Learner English (ICLE) and the \textsc{Trustpilot} corpus, are used to analyze the impact of paraphrasing and demographic factors in the context of NLI tasks. To analyze the effect of paraphrasing on a document, new versions of both corpora are created, which contain paraphrased versions of the documents contained. The effect is inspected using two state-of-the-art NLI systems to perform the task, while the results were analyzed using a regression analysis in combination with dominance analysis (DA). Paraphrasing was found to have a substantial influence in performance of NLI tasks, regardless of corpus, classifier, or paraphrasing method. The usual influence of demographic factors on NLI tasks could not be confirmed in this thesis. Regression analysis and DA allowed for a more profound analysis of the results, which allowed for findings regarding the influence of specific L1s on performance of NLI tasks.Item Open Access Prompt-based continual learning for visual question answering(2024) Ostertag, MagnusIn an ever-evolving world, Continual Learning (CL) strives to enable a costly trained model to learn new tasks without forgetting previously acquired knowledge. This work critically examines current CL benchmarks for Visual Question Answering (VQA), identifying significant shortcomings in the construction introducing bias. To address these issues, we propose a new CL-VQA benchmark based on GQA, designed to be incremental in both the language and the visual modality. Combined with learning it in one modality only, it can offer rich new diagnostics for a model. Additionally, we extend DualPrompt, a prompt-based CL method, DualPrompt, to the multi-modal domain. Using Dark Experience Replay as a baseline, we evaluate the performance against the new benchmark.Item Open Access Evaluating methods of improving the distribution of data across users in a corpus of tweets(2023) Milovanovic, MilanCorpora created from social network data often serve as the data source for tasks in natural language processing. Compared to other, more standardized corpora, social media corpora have idiosyncratic properties due to the fact that they consist of user-generated comments. These are, for example, the unbalanced distribution of the respective comments, a generally lower linguistic quality, and an inherently unstructured and noisy nature. Using a Twitter-generated corpus, I will investigate to what extent the unbalanced distribution of the data has an influence on two downstream tasks, relying on word embeddings. Word embeddings are a ubiquitous and frequently used concept in the field of natural language processing. The most common models are often the means to obtain semantic information about words and their usage by representing the words in an abstract word vector space. The basic idea is that semantically similar words in the mapped vector space have similar vectors. In doing so, these vectors serve as input for standard downstream tasks such as word similarity and semantic change detection. One of the most common models in current research is the use of word2vec, and more specifically, the Skip-gram architecture of this model. The Skip-gram architecture attempts to predict the surrounding words based on the current word. The data on which this architecture is trained greatly influences the resulting word vectors. In the context of this work, however, no significant improvement in the results to a fully preprocessed corpus could be found when filtering methods, widely used in the literature, without specific motivation, are used to select a subset of data according to defined criteria, neither for word similarity nor for semantic change detection. However, comparable results could be achieved with some filters, although the resulting models were trained using significantly fewer tokens as input.Item Open Access Kategorisierung der Zustandsveränderungen bei CoS-Verben auf Basis von Bild- und/oder Textdaten(2023) Godbersen, JuleSowohl textliche als auch visuelle Informationen können für das Verständnis einer Aktion relevant sein. In dieser Bachelorarbeit werden Aktionen betrachtet, die zu einer Veränderung im Zustand des beteiligten Objekts führen. Ziel dabei ist die Beantwortung der Forschungsfrage, welchen Beitrag die Modalitäten bei der Vorhersage von solchen Zustandsveränderungen haben. Die Vorhersage erfolgt mithilfe von Kategorien wie beispielsweise Farbe, Größe und Quantität. Ein wesentlicher Bestandteil dieser Bachelorarbeit ist die Erstellung eines Datensatzes, der Beispielvorkommen von Aktionen mit Zustandsveränderungen enthält. Eine weitere Aufgabe besteht darin, einiger dieser Datenpunkte mit Kategorien von Zustandsveränderungen annotieren zu lassen. Darüber hinaus wird ausgehend von einem visiolinguistischen Modell eine Ablationsstudie durchgeführt. Diese erlaubt mithilfe verschiedener Klassifikatoren, den Einfluss der verschiedenen Modalitäten auf die Leistungsfähigkeit eines Modells im Hinblick auf die Vorhersage von Zustandsveränderungen zu testen. Diese Bachelorarbeit veranschaulicht unter anderem Schwierigkeiten im Rahmen der Annotationen. Die Leistungsfähigkeit bezüglich der Vorhersage von Kategorien, gemessen mit der Akkuratheit, ist bei den Klassifikatoren ähnlich hoch wie bei einem Baseline Modell. Die verschiedenen Klassifikatoren treffen Vorhersagen mit ähnlicher Akkuratheit, sodass die Forschungsfrage mit den Ergebnissen dieser Bachelorarbeit nicht hinreichend beantwortet werden kann. Die Hypothese, dass die Kombination aus textlicher und visueller Modalität komplementäre Informationen liefert und dementsprechend die Kombination beider Modalitäten relevant ist, wird durch die Ergebnisse nicht bestätigt. Ergänzend wird durch diese Bachelorarbeit gezeigt, dass die trainierten Klassifikatoren es ermöglichen, in gewissem Maße auf ungesehene Datenpunkte, ungesehene Verben und ungesehene Domänen zu generalisieren.Item Open Access Cross-lingual word embeddings with multi-sense representations(2024) Shim, Soh-EunCross-lingual word embeddings have been found to be useful in aiding cross-lingual transfer, but work in this line of research has to date rarely addressed the monosemy constraint of static word embeddings in depth, where the collapse of multiple meanings into one form might arguably lead to subpar alignments. In this thesis, we address this gap by examining potential approaches towards the incorporation of sense information into cross-lingual alignment. We explore in specfic two variants of cross-lingual multi-sense alignment: one in which we employ the method of embedding the senses of each word as a Gaussian mixture (Athiwaratkun and Wilson, 2017), where the assumption is that multi-sense embeddings as a basis for alignment may help mitigate the meaning conflation deficiency (Camacho-Collados and Pilehvar, 2018), and in turn help improve isomorphism between vector spaces (Ruder et al., 2019). Our second method explores learning a cross-lingual multi-sense embedding space by reversing the order: we cross-lingually align uni-sense word embeddings, and attempt multi-sense enrichment as a postprocessing step by retrofitting (Pilehvar and Collier, 2016) the embedding on the Open Multilingual Wordnet (Bond et al., 2023). We observe that our model is capable of fine-grained cross-lingual semantic distinctions, where our model successfully identifies colexifications without cross-lingual supervision.Item Open Access Cycle-consistent adversarial networks for automatic speech recognition(2024) Li, Chia-Yu; Vu, Ngoc Thang (Prof. Dr.)Item Open Access An attribution method for classification tasks in Siamese models(2024) Liu, MindongExplaining the contribution of tokens on classification results in the classification task of two sentences is a challenging problem in natural language processing (NLP). This thesis studies the use of the Integrated Jacobians (IJ) in interpreting multi-class classification models with Siamese models, particularly its application in Natural Language Inference (NLI). The NLI task requires models to understand the logical relationships between two sentences, posing challenges for model interpretability. To address the fact that the original Siamese model was primarily designed for regression tasks, the thesis first expanded Siamese models for classification tasks with bilinear similarity while ensuring that the IJ methods can be utilized. It then adapts two forms of the IJ methods: exact IJ and approximate IJ, to work with newly extended Siamese models. To validate the effectiveness of the extended Siamese models using the IJ meth ods, the thesis conducted experiments on the AllNLI dataset under sentence-BERT framework. The thesis employed four different model configurations and applied both IJ methods to these models. The experimental results demonstrate that the IJ methods effectively provide explanations for us. Finally, the thesis examined the consistency between the explanations provided by the IJ methods and semantic relationships at the lexical and span levels using datasets WordNet and SpanEX. In the analysis, the IJ methods show that the models capture semantic relationships between words and spans, and there is a correlation between these relationships and the model’s predictions. This finding supports the use of the IJ methods to explain the decisions of NLP models.Item Open Access Editorial - perspectives for natural language processing between AI, linguistics and cognitive science(2022) Lenci, Alessandro; Padó, Sebastian