Prof. Dr. Lucie Flek at InVirtuo 4.0

In the final session of this semester’s InVirtuo 4.0 seminar, Prof. Dr. Lucie Flek (Lamarr'a Area Chair for NLP) alongside Mahan Akbari Moghanjoughi presented her team’s latest research on enhancing social intelligence in large language models. 

Her work contributes to a growing field focused on ensuring AI systems engage in more natural, meaningful, and socially intelligent conversations—an essential step for the future of human-computer interaction.
 

We focus on three research questions:

RQ1: How should we evaluate perspective taking through the lens of NLG?
We develop a novel evaluation of by asking humans to rank the human response, model output, and a distractor human response, combining approaches from persona consistency. We find that previous consistency evaluation metrics are inadequate and proposed a human ranking evaluation that includes similar human responses. Additionally, we find that our generation model performed competitively with previous work on perspective classification.

RQ2: Do tailored, user-contextualized architectures outperform large language models (LLMs) on this task?
We design two transformer architectures to embed personal context and find that our twin encoder approach outperforms LLMs. We compare tailored architectures to LLMs, including two novel methods, finding that our twin encoder architecture outperformed recent work, FlanT5 and Llama2 models.

RQ3: What user information is most useful to model perspective taking?
Experiments with varied user context showed that self-disclosure statements semantically similar to the conflict situation were most useful.

Read more.

and 

Talk to us during ACL 2024

Prof. Lucie Flek, Dr. Mounika Marreddy and Shangrui Nie look forward to being in Bangkok in August and happy to get in touch!

Data Science and Language Technologies Group @CaisaLab proudly presents two papers: "Do Multilingual Large Language Models Mitigate Stereotype Bias?" and "Perspective Taking through Generating Responses to Conflict Situations" on ACL 2024 in Bangkok and co-organises @HuCLLM Workshop on Thursday, August 15th.

AI24 in Dortmund 

💡 Spotlight on #GenerativeAI at #AI24! Don't miss our essential sessions for professionals at the intersection of AI and industry, highlighting their economic potential on Day 2 of #LamarrCon.

🔎 In the Session “Generative AI – Foundation Models” at 14:45, Prof. Dr. David Jurgens from the University of Michigan will explore "Do Foundation Models have #Personalities?", showcasing the risks and opportunities in aligning these models with specific user groups, critical for personalized AI applications in industries.

Next, at 15:15, Dr. Mehdi Ali from #Lamarr at Fraunhofer IAIS will present "Training Large Language Models at Scale using #Modalities." Learn about their open-source framework for training large-scale models efficiently, a game-changer for businesses looking to leverage AI at scale. 🖥️

In the Session “Generative AI – Large Language Models” at 16:15, Dr. Mounika Marreddy, Vahid Sadiri Javadi, and Dr. Akbar Karimi from the The University of Bonn will lead a hands-on workshop on "#LLMs in Action". Gain practical skills in utilizing state-of-the-art models like Flan-T5 and #Llama, essential for integrating AI into real-world applications. 🔧

These sessions are crucial for understanding how Generative AI can optimize processes, solve complex problems, and create new economic opportunities across various sectors, including logistics and supply chain management. 📈

🎫 Register now to be part of #AI24 – The Lamarr Conference and explore the future of Generative AI: https://lnkd.in/ezG4tsXN

#TechConference | Lucie Flek | OpenGPT-X

Lamarr NLP Researchers Train Multilingual Large Language Models Mitigating Stereotype Bias

Bias in large language models is a well-known and unsolved problem. In our new paper "Do Multilingual Large Language Models Mitigate Stereotype Bias?" we address this challenge by investigating the influence of multilingual training data on model bias reduction.

In the Lamarr NLP research collaboration between Fraunhofer IAIS and the Language Technologies group at the university of Bonn, we have trained six large language models on public data (one for each of Spanish, German, French, Italian and English, and a combined multilingual one), and compared these LLMs to their state-of-the-art counterparts on multilingual bias benchmarks. Our results show that all multilingual models trained on the same number of tokens as monolingual models are less biased for all languages and benchmarks. In addition, our models are generally less biased than selected open-source LLMs of similar size.

We are very happy to announce that our work, conducted by Shangrui Nie, Michael Fromm, Charles Welch, Rebekka Görge, Akbar Karimi, Joan Plepi, Nazia Afsan Mowmita, Nicolas Flores-Herr, Mehdi Ali and Lucie Flek, has been accepted at the C3NLP (Cross-Cultural Considerations in NLP), collocated with the ACL 2024 in Bangkok.  

The challenge of bias in large language models 

Large language models allow us to quickly and easily derive and implement real-world applications in fields as diverse as healthcare, finance and law. Huge amounts of data are used to train these models, resulting in incredible model performance. However, numerous studies have shown that large language models learn biases during training that can lead to discrimination against certain groups of people in their downstream application. Often these biases arise from the training data itself, and they vary between different languages and models.

Multilinguality as a solution approach

To avoid the harm of discrimination, research aims to mitigate bias in large language models. Among various approaches, previous research indicates that using multilingual training data to train large language models reduces model bias. In doing so, the models can benefit from the use of different languages, which differ in semantics and syntax and cover a wider cultural diversity. Within our work, we are building up on this previous work by investigating especially in the impact of monolingual versus multilingual data of larger, decoder-based language models. 

Our experiments and findings 

In our experiments, we train six novel large language models, one each for Spanish, German, French, Italian and English, as well as a multilingual model trained on all five languages but using the same number of tokens. To compare the bias of the monolingual and multilingual models, we benchmark them against two well-known bias evaluation benchmarks CrowS-Pairs andBBQ. The prior measures the degree to which a model prefers stereotyping over a less stereotyping sentences, while the latter is a question-answering dataset that requires from a model to answer stereotype-reflecting questions regarding two social groups (compare Figure 1).  As both benchmarks are originally only available in English, we perform a human-validated automated translation. 

Our results support the initial hypothesis that multilingual large language models reduce model bias. Within the experiments, we find that all multilingual models trained on the same number of tokens as monolingual models are less biased than the monolingual models for all languages and both benchmarks. We also find that our models are generally less biased than selected open-source large language models of similar size, although they fall short of zero-shot prompt-based approaches with GPT3.

Read more. 
 

“Relate – Talk with Me”

The interactive exhibition by Andrea Familari is open until February 2, 2025.

This installation examines the intersection of Artificial Intelligence and human relationships, allowing visitors to converse with a custom voice bot represented by a lifelike avatar.

Prof. Lucie Flek, Area Chair for Natural Language Processing at the Lamarr Institute, contributed her expertise to enhance the bot's conversational abilities, tackling issues of bias and the dynamics of human-AI interaction. At the Lamarr Institute, we are dedicated to advancing AI research with a focus on its societal impact, and projects like “Relate – Talk with Me” reflect our mission to explore AI's role in communication and connection.

Professor Dr Lucie Flek accompanies NRW Minister President Hendrik Wüst on a delegation trip to the USA

Prof. Lucie Flek representing the University of Bonn and the Lamarr Institute speaking on "Transatlantic AI Symposium: Europe's Heartbeat of Innovation" hosted by NRW.Global Business, trade and investment agency of the German state of North Rhine-Westphalia (NRW).

Cooperation and coordination of leading researchers in Europe and USA is the way how to cope with complex AI challenges and utilitize new AI opportunities. Read more.