Research Groups

LARGE LANGUAGE MODELS (Seminar)

Winter Semester 2023 – 2024

Content:

What is the Large Language Models seminar about?

Large Language Models (LLMs), such as GPT-3, BERT, and their successors, have had an enormous impact on various domains, including natural language processing, machine learning, and artificial intelligence. These models have redefined what’s possible in applications such as text generation, translation, summarization, sentiment analysis, and more. The aim of this seminar is to explore the cutting-edge research, insights, and trends in the field of LLMs.

Logistics:

Seminars: are on Tuesday 10:00 AM - 11:30 AM in B-IT 2.113 (Friedrich-Hirzebruch-Allee 6). ZOOM LINK

Note: The class is hybrid, it is fine to attend/ present online, but be ready to share your screen for the hands-on results!

Course Materials: will be uploaded every week on eCampus.

Contact: Students should ask all course-related questions in our forum discussion on eCampus. For external inquiries, emergencies, or personal matters, you can contact Prof. Flek or Vahid.

Office Hours: Please reach out to us first via mail to arrange any in-person meeting.
- Prof. Dr. Lucie Flek: Friedrich-Hirzebruch-Allee 6 (B-IT) – Room: 2.123
- Vahid Sadiri Javadi: Friedrich-Hirzebruch-Allee 6 (B-IT) – Room: 2.120

NEWS / UPDATES:

24.10.2023: The first class starts on Tuesday, 24.10.2023 at 10:00 AM

Instructors:

Prof. Dr. Lucie Flek

flek(at)bit.uni-bonn.de

Head of CAISA Lab

Vahid Sadiri Javadi

vahidsj(at)bit.uni-bonn.de

Course Coordinator

Seminar Work:

1. Presentation:

A group of 2-3 people presents every week on a selected topic:
- You summarize a paper or a set of papers in a presentation
- You showcase your point with a model API or web interface
- You prepare a short hands-on session for the group as a part of your presentation (can others fool / hack/ break / improve the LLMs in the aspect you discuss?)

2. Final Assignment:

To complete the course, you need to complete a final assignment:
- In addition to the group presentation, you need to create an evaluation dataset on a challenging LLM problem (e.g. commonsense reasoning, perspective taking, cross-lingual QA, stereotype bias, etc.)
- Instead of writing a 3000-word essay, you “write” a dataset of ca. 3000 words (ca. 300 sentences) and evaluate 3 open LLMs on it
- The dataset cannot be LLM-generated
- More people can work on the same topic to create a larger dataset
- The language(s) of the data can be freely chosen
- The creation process and the author's background need to be documented (see [1] and [2])

Deadlines:
- Block your presentation slot until: 31.10.2023
- Register your assignment plan until: 12.12.2023
- Hand in your assignment until: 30.01.2024

Submission: Presentations and Final Assignments should be submitted via eCampus. Further instructions will be announced soon. Please do not email us your assignments.

Allocation:

3 + 1 SWS
Master in Media Informatics: 4 ECTS credits
Students must register for the exam on POS/BASIS.

Literature:

Bommasani, Rishi, et al., 2021, arXiv preprint: On the opportunities and risks of foundation models
Devlin, Jacob, et al., 2018, arXiv preprint: Bert: Pre-training of deep bidirectional transformers for language understanding
Brown, Tom, et al., 2020, arXiv preprint: Language models are few-shot learners
WX Zhao, et al., 2023, arXiv preprint: A survey of large language models
Yang, Jingfeng, et al., 2023, arXiv preprint: Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond
Awesome Foundation Models: GitHub Repository

Week	Date	Description	Events	Deadlines
Week 0		Organization & Outline
Week 1		Introduction to LLMs What exactly is generative AI? How do LLMs work? [3] Differences in LLM architectures Main contributions of LLMs to the field (Why/how did this happen?) Open challenges (What still doesn’t work and what LLMs are not made for)
Week 2		Do LLMs work? LLM bias issues Robustness Alignment Evaluation
Week 3
Week 4
Week 5
Week 6