Block Seminar Machine Learning for Language Processing (Fall 2026)

Deep learning is the predominant machine learning paradigm in language processing. This approach not only gave huge performance improvements across a large variety of natural language processing tasks.

This spring, we will look into transformer language models and how they can integrate graph information.

Lecturer:  Dietrich Klakow

Location:  t.d.b

Time: block course in the fall break 2026 however preparations start earlier. Here the specific time line

Closing topic doodle: tbd
Kick-Off: tbd
One page outline: tbd
Draft presentation: tbd

Practice talks and final talks will be during the spring break. Time/date will be decided during the kick-off.

Application for participation:  see CS seminar system (for CS; BioInf, DSAI, VC, ES, …), for CoLI,LST, LCT use LSF to apply.

HISPOS registration deadline: Friday, July 10th

Grading (tentative):

  • 5% one page talk outline
  • 10% draft presentation
  • 10% practice talk
  • 25% own experiments and coding
  • 10% report on coding task
  • 35% final talk
  • 5% contributions to discussion during final talk of fellow participants

List of Topics (tentative):

  1. Entropy in Large Language Models
  2. Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
  3. Fundamental limits of overparametrized shallow neural networks for supervised learning
  4. Entropy and mutual information in models of deep neural networks
  5. Information-Theoretic Generalization Bounds for Deep Neural Networks 
  6. A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks
  7. On the Spectral Bias of Neural Networks
  8. Deep Learning Foundation Models from Classical Molecular Descriptors
  9. Geodiff: A geometric diffusion model for molecular conformation generation
  10. Optimization of molecules via deep reinforcement learning
  11. Toward Closed-loop Molecular Discovery via Language Model, Property Alignment and Strategic Search
  12. Equivariant Diffusion Models for Molecules
  13. MolCA: Molecular Graph-Language Modelling with Cross-Modal Projector and Uni-Modal Adapter
  14. Graph2Token – Bridging Molecular Graphs and Large Language Models
  15. CL-MFAP: A Contrastive Learning-Based Multimodal Foundation Model for Molecular Property Prediction and Antibiotic Screening