05 Fakultät Informatik, Elektrotechnik und Informationstechnik

Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6

Browse

Publication Type: search.filters.UBSTyp.Abschlussarbeit (Master)Subject: search.filters.subject.004Subject: search.filters.subject.400Start date: 2024

Search Results

Now showing 1 - 10 of 27
  • Thumbnail Image
    ItemOpen Access
    Tailoring language for user backgrounds : human and LLM evaluations of paraphrased texts
    (2025) Carlon, Francesca
    In an increasingly interdisciplinary world, the ability to tailor technical and field-specific language to diverse audiences has become essential. Both academic research and industry benefit from the knowledge and expertise of individuals from varied technical backgrounds, supporting progress in research and product development. Therefore, it has become essential to make domain-specific concepts accessible and understandable to different audiences. This study investigates how Large Language Models (LLMs) can generate effective paraphrases by tailoring language to specific target audiences. It also explores which linguistic elements enhance conceptual understanding and whether LLMs can produce high-quality synthetic data for evaluating text quality. This research specifically focuses on tailoring language from Linguistics, Computer Science, and Computational Linguistics texts for audiences with diverse academic backgrounds, using a variety of LLMs. Methods employed include Prompt Engineering, linguistic metrics, and evaluations conducted by both human judges and LLM-as-a-judge approaches. The experiments focus on assessing prompt quality, examining the relationship between human evaluations and linguistic features, and analyzing the correlation between human judgments and LLM-as-a-judge assessments. The results demonstrate that combining Role Prompting with Chain-of-Thought (CoT) prompting effectively produces high-quality paraphrased texts. Additionally, syntactic and lexical features play a central role in enhancing text accessibility and users’ understanding of unfamiliar concepts. Finally, while the correlation between LLM-as-a-judge evaluations and human judgments is promising, it varies across different user groups. Therefore, this research demonstrates the potential of Prompt Engineering techniques for paraphrasing and tailoring language to users with different backgrounds, explores the linguistic features that enhance text comprehension, and examines the extent to which LLM-as-a-judge correlates with human preferences and can support the automatic retrieval of high-quality data.
  • Thumbnail Image
    ItemOpen Access
    Cross-lingual word embeddings with multi-sense representations
    (2024) Shim, Soh-Eun
    Cross-lingual word embeddings have been found to be useful in aiding cross-lingual transfer, but work in this line of research has to date rarely addressed the monosemy constraint of static word embeddings in depth, where the collapse of multiple meanings into one form might arguably lead to subpar alignments. In this thesis, we address this gap by examining potential approaches towards the incorporation of sense information into cross-lingual alignment. We explore in specfic two variants of cross-lingual multi-sense alignment: one in which we employ the method of embedding the senses of each word as a Gaussian mixture (Athiwaratkun and Wilson, 2017), where the assumption is that multi-sense embeddings as a basis for alignment may help mitigate the meaning conflation deficiency (Camacho-Collados and Pilehvar, 2018), and in turn help improve isomorphism between vector spaces (Ruder et al., 2019). Our second method explores learning a cross-lingual multi-sense embedding space by reversing the order: we cross-lingually align uni-sense word embeddings, and attempt multi-sense enrichment as a postprocessing step by retrofitting (Pilehvar and Collier, 2016) the embedding on the Open Multilingual Wordnet (Bond et al., 2023). We observe that our model is capable of fine-grained cross-lingual semantic distinctions, where our model successfully identifies colexifications without cross-lingual supervision.
  • Thumbnail Image
    ItemOpen Access
    An attribution method for classification tasks in Siamese models
    (2024) Liu, Mindong
    Explaining the contribution of tokens on classification results in the classification task of two sentences is a challenging problem in natural language processing (NLP). This thesis studies the use of the Integrated Jacobians (IJ) in interpreting multi-class classification models with Siamese models, particularly its application in Natural Language Inference (NLI). The NLI task requires models to understand the logical relationships between two sentences, posing challenges for model interpretability. To address the fact that the original Siamese model was primarily designed for regression tasks, the thesis first expanded Siamese models for classification tasks with bilinear similarity while ensuring that the IJ methods can be utilized. It then adapts two forms of the IJ methods: exact IJ and approximate IJ, to work with newly extended Siamese models. To validate the effectiveness of the extended Siamese models using the IJ meth ods, the thesis conducted experiments on the AllNLI dataset under sentence-BERT framework. The thesis employed four different model configurations and applied both IJ methods to these models. The experimental results demonstrate that the IJ methods effectively provide explanations for us. Finally, the thesis examined the consistency between the explanations provided by the IJ methods and semantic relationships at the lexical and span levels using datasets WordNet and SpanEX. In the analysis, the IJ methods show that the models capture semantic relationships between words and spans, and there is a correlation between these relationships and the model’s predictions. This finding supports the use of the IJ methods to explain the decisions of NLP models.
  • Thumbnail Image
    ItemOpen Access
    Bridging behavioral gaps : automatic extrapolation of concreteness norms in Arabic using English-tuned KNN approaches
    (2025) Aziz, Marina
    This thesis addresses the automatic extrapolation of concreteness norms for nouns in both Modern Standard Arabic and English. The main goal is to enable reliable estimation of how concrete or abstract words are (e.g. “apple” vs. “justice”) using computational methods, supporting applications in psycholinguistics and natural language processing. To this end, a novel dataset of 202 Arabic nouns rated for concreteness is introduced and aligned with established English norms. To predict concreteness, the study compares a K-Nearest Neighbors (KNN) regression model based on FastText and transformer-based embeddings with predictions from Chat GPT. The KNN models achieve high accuracy, Spearman ρ = 0.92 (RMSE = 0.43) for English and ρ = 0.83 (RMSE = 0.69) for Arabic on held-out test sets. By contrast, ChatGPT predictions, while consistent across runs, yield lower correlations (ρ = 0.80 for both English and Arabic) and higher RMSE values, confirming that KNN remains more accurate for concreteness estimation. Keywords: concreteness norms; abstractness; Arabic; English; K-Nearest Neighbors; word embeddings; FastText; transformer models; ChatGPT; lexical semantics; psycholinguistics; norm extrapolation
  • Thumbnail Image
    ItemOpen Access
    Melodies of the majority language : investigating prosodic phenomena in majority English
    (2025) Quraishi, Tanishq
    This thesis is a comparative study of majority and monolingual English speaker groups on three key prosodic phenomena, namely pitch accents, boundary tones and intonational phrases. To that end, the data employed is extracted from a multilingual corpus consisting of comparable speaker data with the necessary annotations. The nature of the dataset is on of semi-spontaneous narratives elicited in a semi-naturalistic manner. Three binomial generalized linear mixed effects models and one linear mixed effect model encode fixed and random effects in order to analyse the influence and interaction of various factors such as bilingualism, gender and formality on linguistic intonation. The statistical analysis performed on semi-spontaneous speech, reveals a significant influence of formality on higher monotonal pitch accent usage. The influence of gender is significant on the likelihood of a speaker using a high boundary tone in their utterance. The role of bilingualism and formality influence pitch accent placements on content words, and lastly formality impacts the length of an intonational phrase, where formal contexts consist of shorter phrases.
  • Thumbnail Image
    ItemOpen Access
    Decoding the Baroque : development of a novel dataset for transformer-based harmonic analysis of flute music
    (2025) Cunningham, Andrea Maria
    The computational modeling of functional harmony in the field of Music Information Retrieval (MIR) faces significant challenges when addressing the intricate structural relationships inherent in historical styles like Baroque music. This research introduces a novel dataset of Baroque flute sonatas with detailed functional harmony annotations spanning church and chamber traditions. A specialized Transformer-based model for automatic melody harmonization is developed and evaluated on its ability to generate stylistically appropriate harmonizations across composers and forms. The study proposes that a segmentation-aware Transformer model with innovative attention mechanisms can produce stylistically coherent harmonizations while revealing tensions between period conventions and composer-specific idioms. By examining Baroque flute instrumental music beyond Bach chorales, this study addresses a critical gap in computational musicology and demonstrates how sequence-to-sequence architectures capture hierarchical structures analogous to linguistics. This thesis describes the data curation process of the Baroque flute sonata dataset, presents the enhanced Transformer model, outlines the methodology, and analyzes results concerning harmonic accuracy, stylistic coherence, and musicological implications.
  • Thumbnail Image
    ItemOpen Access
    Using retrieved augmented generation for question answering with LLMs in the cybersecurity domain
    (2025) Moxon, Linnet
    This work investigates the impact of Retrieved Augmented Generation on the performance of Large Language Models in a Question Answering scenario in the cybersecurity domain. 14 different context setups are built, based on the two data resources NIST and MITRE ATT&CK, and retrieved with tf-idf retrieval. The questions from Tihanyi et al. (2024) are prompted with and without context, and with and without answer options. The evaluation shows that the context only partially increases the performance of the three models meta.llama3-1-8b-instruct-v1:0, mistral.mixtral-8x7b-instruct-v0:1 and mistral.mistral-7b-instruct-v0:2. Further, there are differences observed in how well the models follow the prompt instructions, which clearly impacts the findings. In the future a more advanced retrieval method, different data resources for building the databases and a better alignment of the prompt length, could positively influence the results.
  • Thumbnail Image
    ItemOpen Access
    RAGAR, your falsehood RADAR : RAG-augmented reasoning for political fact-checking using multimodal large language models
    (2024) Abdul Khaliq, Mohammed
    The escalating challenge of misinformation, particularly in the context of political discourse, necessitates advanced solutions for fact-checking. This thesis introduces innovative approaches to enhance the reliability and efficiency of multimodal fact-checking through the integration of large language models (LLMs) with Retrieval-augmented Generation (RAG) based advanced reasoning techniques. In the digital era, where misinformation spreads rapidly across various media, including text and images, there's a critical need for robust mechanisms capable of evaluating the veracity of political claims. This work proposes two novel methodologies, Chain of RAG (CoRAG) and Tree of RAG (ToRAG), and their hybrid implementations incorporating Chain of Thought and Chain of Verification. These approaches leverage RAG techniques utilizing multimodal LLMs with reasoning techniques. The approaches are designed to process and assess political claims by considering textual and visual information, providing a comprehensive approach to fact-checking. This thesis explores the implementation of these approaches within a multimodal fact-checking pipeline, highlighting their effectiveness in improving the accuracy of veracity predictions and the generation of explanations. By employing multimodal LLMs adept at analyzing text and images, this research advances the capability of automated systems in identifying and countering misinformation. The experimental evaluation demonstrates that the proposed RAG-augmented Reasoning (RAGAR) techniques outperform existing methods that rely on sub-question generation, offering a promising solution to the challenges of political fact-checking. This thesis contributes to the fields of computational linguistics and political science by providing an effective approach to combat fake news, thereby enhancing the integrity of political discourse in the digital age.
  • Thumbnail Image
    ItemOpen Access
    Exploring the effects of enriched English language input on language model efficiency
    (2024) Zeller, Tom
    Recent years have seen the advent of large-scale language modeling as exemplified by transformer-based models like GPT or variants of the BERT architecture. These models, which are trained on massive datasets and using compute unattainable by actors that are not of the scale of the biggest tech companies, have shown impressive feats of syntactic and semantic understanding. Naturally, interest has risen in making these models more efficient, in terms of compute as well as data requirements. Research in this area can be seen as primarily motivated by two factors: reducing the barrier for smaller actors like research institutes or end consumers to train and execute state-of-the-art models, as well as reducing the carbon footprint of these models. To achieve this goal, model compression techniques like quantization, pruning or distillation are utilized. This work aims to explore a different, less model-centric and more data-centric approach: Modifying the training and inference data, by enriching it with syntactic and semantic information. To this end, a lexical resource is created which maps English words to a form where individual characters represent values of a range of semantic and syntactic features, providing lexical information that is accessible to all model types that operate on tokens at the sub-word or character-level. Different features and methods of representation are discussed, and their effect on model performance is evaluated by pretraining a small GPT-family model and fine-tuning on downstream tasks of the SuperGLUE benchmark. Given a fixed amount of data and compute, the experiments show a performance advantage for a character-level model trained using the enriched data.
  • Thumbnail Image
    ItemOpen Access
    More reliable retrieval augmented generation for domain-specific question-answering through domain-infused soft prompts
    (2025) Nassar, Zeina
    Transformer-based large language models (LLMs) have revolutionized the field of artificial intelligence, enabling advancements in various applications due to their exceptional reasoning capabilities. However, these models often face challenges such as hallucination, outdated knowledge, and non-transparent reasoning, which limit their reliability in critical tasks like question answering (QA). QA systems, especially in domain-specific contexts like car manuals, require high accuracy and reliability to ensure user safety. Achieving this is complicated by the need for precise retrieval and interpretation of domain-specific information, which can be hindered by unfamiliar keywords or ambiguous questions. Open-domain QA relies on external knowledge repositories and typically follows a retriever-reader framework to locate and process evidence. In contrast, domain-specific QA often lacks sufficient gold-standard datasets, making it essential to explore techniques like retrieval-augmented generation (RAG), which combines retrieved context with the question as input to LLMs. While RAG improves grounding, hallucinations still occur when models rely on pre-trained knowledge over retrieved evidence. This thesis investigates the role of prompting techniques in improving faithfulness and reducing hallucinations in domain-specific QA. By testing domain-specific and domain-agnostic discrete prompts as well as soft prompting methods, this work aims to identify strategies for generating more accurate and grounded responses. The study addresses key research questions on the effectiveness of domain-specific information in prompts, the best way to incorporate such information, and whether dynamic soft-prompts outperform static ones in domain-specific QA scenarios. The findings aim to contribute to building more reliable and factual QA systems.