Block course
Time & Location: kick-off meeting in April-May; presentation meetings indicatively 2-3 last weeks of September or 2-3 first weeks of October
Teacher: Dr Volha Petukhova
*** Announcements***
Registration in LSF
Join TEAMS
Kick-off: TBA
Kick-off & Introduction slides: see TEAMS Class Material
Suitable for: CoLi, CS and CuK
Organization:
We plan to hold a first planing meeting early in the semester. For the actual seminar (doodle decision on time and papers) we will have a talk for each participant of 30 minutes followed by 10 minutes discussion (discussions participation will be also graded) . After the talk, the presenter has to prepare a short about 10 pages report and hand it in for grading.
Grading: 40% based on the talk, 40% based on the report, 20% based on discussions participation.
Term paper:
Topics:
Understanding and generation of multimodal human dialogue behavior;
Social signals/affective computing;
Multimodal dialogue modelling;
Large Language Models for Dialogue Modelling and Analysis
Multimodal dialogue systems & applications
*Each talk will be based on a research paper
Multimodality: multimodal behaviour, annotations and tools
1. Saberi, M., DiPaola, S., & Bernardet, U. (2021). Expressing Personality Through Non-verbal Behaviour in Real-Time Interaction. Frontiers in Psychology, 12, 660895.
2. Voß, H., & Kopp, S. (2023, September). Augmented Co-Speech Gesture Generation: Including Form and Meaning Features to Guide Learning-Based Gesture Synthesis. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents.
3. Xu, S., Fukumoto, F., Go, J. L. K., & Suzuki, Y. (2023). Learning Disentangled Meaning and Style Representations for Positive Text Reframing. INLG 2023, 424.
4. Hartholt, Arno, Ed Fast, Zongjian Li, Kevin Kim, Andrew Leeds, and Sharon Mozgai. (2022, September) Re-architecting the virtual human toolkit: towards an interoperable platform for embodied conversational agent research and development. In Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents.
Emotions, social signals and interaction
5. Joby, N. E., & Umemuro, H. (2023, September). Emotional mimicry as a proxy measurement for pro-social indicators of trust, empathy, liking and altruism. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents
6. Schneeberger, T., Reinwarth, A. L., Wensky, R., Anglet, M. S., Gebhard, P., & Wessler, J. (2023, September). Fast Friends: Generating Interpersonal Closeness between Humans and Socially Interactive Agents. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents
7. Welivita, A., Yeh, C. H., & Pu, P. (2023, September). Empathetic response generation for distress support. In Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue (pp. 632-644).
8. Feng, S., Lubis, N., Ruppik, B., Geishauser, C., Heck, M., Lin, H. C., … & Gašić, M. (2023). From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue. arXiv preprint arXiv:2308.12648.
9. Antunes, A., Campos, J., Guimarães, M., Dias, J., & Santos, P. A. (2023, September). Prompting for Socially Intelligent Agents with ChatGPT. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents.
Multimodal fusion, dialogue modelling and management
10. Hirano, Y., Okada, S., & Komatani, K. (2021, October). Recognizing Social Signals with Weakly Supervised Multitask Learning for Multimodal Dialogue Systems. In Proceedings of the 2021 International Conference on Multimodal Interaction (pp. 141-149).
11. Christopher Hidey, Fei Liu, and Rahul Goel. (2022, September). Reducing Model Churn: Stable Re-training of Conversational Agents. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 14–25, Edinburgh, UK. Association for Computational Linguistics.
12. Richardson, C., Sundar, A., & Heck, L. (2023). SYNDICOM: Improving Conversational Commonsense with Error-Injection and Natural Language Feedback. arXiv preprint arXiv:2309.10015.
13. Raposo, G., Coheur, L., & Martins, B. (2023, September). Prompting, Retrieval, Training: An exploration of different approaches for task-oriented dialogue generation. In Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue (pp. 400-412).
14. Coca, A., Tseng, B. H., Chen, J., Lin, W., Zhang, W., Anders, T., & Byrne, B. (2023). Grounding Description-Driven Dialogue State Trackers with Knowledge-Seeking Turns. arXiv preprint arXiv:2309.13448.
15. Ramirez, A., Agarwal, K., Juraska, J., Garg, U., & Walker, M. A. (2023). Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking. arXiv preprint arXiv:2307.14440.
Large Language Models for Dialogue Modelling and Analysis
16. Hudeček, V., & Dušek, O. (2023, September). Are Large Language Models All You Need for Task-Oriented Dialogue?. In Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue (pp. 216-228).
17. Bertolazzi, L., Mazzaccara, D., Merlo, F., & Bernardi, R. (2023, September). ChatGPT’s Information Seeking Strategy: Insights from the 20-Questions Game. In Proceedings of the 16th International Natural Language Generation Conference (pp. 153-162).
18. Finch, S. E., Paek, E. S., & Choi, J. D. (2023). Leveraging large language models for automated dialogue analysis. arXiv preprint arXiv:2309.06490.
19. Addlesee, A., Sieińska, W., Gunson, N., Garcia, D. H., Dondrup, C., & Lemon, O. (2023). Multi-party goal tracking with LLMs: Comparing pre-training, fine-tuning, and prompt engineering. arXiv preprint arXiv:2308.15231.
20. Ostyakova, L., Smilga, V., Petukhova, K., Molchanova, M., & Kornev, D. (2023, September). ChatGPT vs. Crowdsourcing vs. Experts: Annotating Open-Domain Conversations with Speech Functions. In Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue (pp. 242-254).
Multimodal dialogue systems & applications
21. Marcoux, A., Tessier, M. H., & Jackson, P. L. (2023, September). Nonverbal Markers of Empathy in Virtual Healthcare Professionals. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents (pp. 1-4).
22. Cherakara, N., Varghese, F., Shabana, S., Nelson, N., Karukayil, A., Kulothungan, R., … & Lemon, O. (2023). FurChat: An Embodied Conversational Agent using LLMs, Combining Open and Closed-Domain Dialogue with Facial Expressions. arXiv preprint arXiv:2308.15214.
23. Garcia, J. C., Suglia, A., Eshghi, A., & Hastie, H. (2023, July). ‘What are you referring to?’Evaluating the ability of multi-modal dialogue models to process clarificational exchanges. In 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue (pp. 1-8). Association for Computational Linguistics.
24. Shoa, A., Oliva, R., Slater, M., & Friedman, D. (2023, September). Sushi with Einstein: Enhancing Hybrid Live Events with LLM-Based Virtual Humans. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents (pp. 1-6).
For any questions, please send an email to:
v.petukhova@lsv.uni-saarland.de
Use subject tag: [MDS_2024]