Elective for CS graduate students at the Technische Hochschule Nürnberg.

Class Schedule and Credits

Time and Location:

Announcements and Discussions: Join on Teams wih code: endtu0s.

Each week, we will discuss algorithms and their theory before implementing them to get a better hands-on understanding. The materials consist of a mix of required and recommended readings, slides as well as a set of mandatory programming and analysis assignments. Pair programming encouraged, BYOD strongly recommended!

Assignments

Lectures are accompanied by mandatory assignments that will be reviewed and discussed every week. All assignments are in form of prepared python3 jupyter notebooks. Depending on the topic, they consist of programming, evaluation and/or discussion sections. Please submit the notebooks (including state/rendered cells) the Teams channel under Files > Assignment Submissions (see schedule below). Pair-programming is encouraged, but everyone needs to submit an individual file.

Credits

Credits are earned through two components:

  1. All assignments must be completed on time (see schedule below; pass/fail).
  2. Oral exam (20’) covering theory and assignments (graded; individual exams).

Note: Materials will be (mostly) in English, the lectures/tutorials will be taught in German unless English speaker present; oral exam in language of choice.

Please note the required reading remarks in the syllabus.

Schedule

Date Topic Required Reading Materials Assignment due
Mar 17 no class      
Mar 18 comparing sequences: ED/Levenshtein, NW, DTW, modeling cost; intro A1 Chao/Zhang Ch. 1.2 through 1.4, 2.4 and 3. introduction, comparing sequences  
Mar 24 no class      
Mar 25 Assignment 1   A1 Dynamic Programming  
Mar 31 Markov chains: statistical modeling of discrete sequences; discussion A1, intro A2 Schukat-Talamazzini Ch. 7.2.{1,2}, 7.3 markov chains  
Apr 1 Assignment 2   A2 Markov chains  
Apr 7 HMMs, pt. 1: basics, BW, time-alignments; intro A3.1; Schukat-Talamazzini Ch. 5 & 8 Hidden Markov Models A1
Apr 8 review A2; Assignment 3, pt. 1   A3 HMM  
Apr 14 HMMs, pt. 2: Viterbi, beam-decoding, higher order modeling; intro A3.2 Decoding curtesy of Elmar Nöth   A2
Apr 15 Assignment 3, pt. 2   A3 HMM  
Apr 21 no class (Easter)      
Apr 22 no class (Easter)      
Apr 28 nnets 1: fundamentals, FF, AE, Word2Vec, FastText, ConvNets, embeddings, HMM-DNN; intro A4 Karpathy 2016: Yes you should understand backprop, Mikolov et al., 2013. Efficient Estimation of Word Representations in Vector Space, Waibel et al., 1989. Phoneme Recognition Using Time-Delay Neural Networks, LeCun et al., 1998. Gradient-based learning applied to document recognition nnets basics, nnets pt.1 A3
Apr 29 Assignment 4   A4 nnets  
May 5 nnets2: RNN, LSTM, GRU; intro A5 Pytorch Seq2Seq Tutorial, Pascanu et al. On the difficulty of training recurrent neural networks, Chris Olah Understanding LSTM Networks slides  
May 6 no class      
May 12 nnets3: s2s, Attn, CTC, RNN-T; intro A6 Graves et al., 2006. Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks, Loren Lugosch’s Introduction to RNN-T, Graves et al., 2012. Sequence Transduction with Recurrent Neural Networks attention, ctc, rnn-t transfer learning A4
May 13 Assignment 5   A5 RNN  
May 19 tranformers: basics, architectures for text (BERT, SBERT, GPT) and speech (Wav2Vec2); intro A8 Łukasz Kaiser: Attention is all you need, esp. at 15:45ff. Vaswani et al. Attention Is All You Need, Jay Alammar The Illustrated Transformer, Devlin et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, wav2vec: Unsupervised Pre-training for Speech Recognition, Radford et al. Improving Language Understanding by Generative Pre-Training attention, ctc, rnn-t transfer learning A5
May 20 Assignment 6   A6 attention  
May 26 LLMs as foundation models: benchmarks, GPT, BPE, data (pretraining and fine-tuning), instructGPT, RLHL; intro A9; conditioning of transformers: zero-/few-shot, CoT road to gpt   A6
May 27 Assignment 7   A7 Transformers  
Jun 2 no class (BV IN-2021-01)      
Jun 3 Assignments (cont’d)      
Jun 9 no class (Pentecost)      
Jun 10 no class (Pentecost)      
Jun 16 no class (summer school)      
Jun 17 no class (summer school)      
Jun 23 reinforcement learning reinforcement learning   A7
Jun 24 Assignment 8   A8 Summarization  
Jun 30 state space models (tbd)     A8
Jul 1 wrap-up      
Calendar Week 28 Exam week      

Subscribe to https://github.com/sikoried/sequence-learning/ to follow updates.