Toggle navigation
JUPYTER
FAQ
View as Slides
View as Code
Python 3 Kernel
View on GitHub
Execute on Binder
Download Notebook
stat-nlp-book
open_items.ipynb
Notebook
$$ \newcommand{\Xs}{\mathcal{X}} \newcommand{\Ys}{\mathcal{Y}} \newcommand{\y}{\mathbf{y}} \newcommand{\balpha}{\boldsymbol{\alpha}} \newcommand{\bbeta}{\boldsymbol{\beta}} \newcommand{\aligns}{\mathbf{a}} \newcommand{\align}{a} \newcommand{\source}{\mathbf{s}} \newcommand{\target}{\mathbf{t}} \newcommand{\ssource}{s} \newcommand{\starget}{t} \newcommand{\repr}{\mathbf{f}} \newcommand{\repry}{\mathbf{g}} \newcommand{\x}{\mathbf{x}} \newcommand{\prob}{p} \newcommand{\vocab}{V} \newcommand{\params}{\boldsymbol{\theta}} \newcommand{\param}{\theta} \DeclareMathOperator{\perplexity}{PP} \DeclareMathOperator{\argmax}{argmax} \DeclareMathOperator{\argmin}{argmin} \newcommand{\train}{\mathcal{D}} \newcommand{\counts}[2]{\#_{#1}(#2) } \newcommand{\length}[1]{\text{length}(#1) } \newcommand{\indi}{\mathbb{I}} $$
Open Items Lecture 2
¶
Fixed
shirtless slide
!
Workload for assignments: 5h-20h per assignment, depending on your background, skills etc
Module Selection Priorities:
Optional
Elective
Non-CS (update: will not be impossible)
First Come First Serve, but Elective students will only know if they have a spot after next Friday
Open Items Lecture 3
¶
Please use the moodle for questions regarding the assignment
No office hour today
Next:
Language Models: MLE
Open Items Lecture 4
¶
Assignment questions?
Please use the moodle for questions regarding the assignment
I will not be available for office hours today
Next:
Language Models: MLE
Open Items Lecture 5
¶
Assignment Questions
Are we allowed to use the
replace_OOVs
function on
_snlp_test_song_words
?
Will the hidden test set we are assessed on already have the replace OOV's function applied to it?
Can we use helper functions from the exercices in the coursework?
Question on the
OOVAwareLM
Next: Word-Based Machine Translation
Open Items Lecture 6
¶
Do read the course notes
Assignment:
You are allowed to use both development and training set as "larger training set" but understand the potential drawbacks.
If you try something interesting that doesn't work, do report it!
There is a potential to cheat (don't)
Re question on simplification:
http://nlp.cs.swarthmore.edu/semeval/tasks/task10/summary.shtml
Next: MT Noisy Channel
Open Items Lecture 7
¶
LectureCast: online!
Next:
Word-based MT
, EM Algorithm
Open Items Lecture 8
¶
Assignments:
It's possible to train most (unless neural) LMs by scanning through the data once and gathering counts
You often do not need to calculate normalizers by explicit summing (e.g. in Absolute Discounting)
Next: CFGs in
Parsing
Open Items Lecture 9
¶
Assignment 2 coming out today!
will be about feature engineering
see lecture notes on text classification and sequence labelling
you have 4 weeks!
After reading week we will discuss possible MSc projects
Next:
Parsing
, PCFG
Open Items Lecture 10
¶
South England NLP Meetups
Next: text classification
Open Items Lecture 11
¶
Office hours: now at 90 High Holborn, still Friday 5 PM - 6 PM
Dec 6 & 8: Pontus will present
Last Week: Industry Speakers (currently: DeepMind, Benevolent AI)
Next: Conditional Log-Linear Models
Open Items Lecture 12
¶
Please fill out the attendance form
Assignment 1 Marking:
Students did change the "do not change" fields
Students did submit code that didn't work (I will get in touch with you separately)
Assignment 2 Typo: You get 20pts for Task 1!
Office hours: now at 90 High Holborn, still Friday 5 PM - 6 PM
But I will head over to the cafeteria in the Roberts building to chat for a while
Next: Sequence Models
Open Items Lecture 13
¶
Assignment 2: Annotation Noise?
Next: Maximum Entropy Markov Models
Open Items Lecture 14
¶
Assignment 2: You can't save/submit your training parameters, but you can fix your hyperparameters
Thanks for amazing support on Moodle!
Next: Unsupervised Relation Extraction
Open Items Lecture 15
¶
Assignment 1: Marks out today.
Scores for description come with short justifications based on a global point system
Creativity vs Substance: KN standard on creativity, high on substance
Assignment 2: Due today!
Assignment 3: coming out today
Next: Dependency Parsing
Open Items Lecture 16
¶
Assignment 1 Marks
Please be polite when requesting re-assessment or raise issues
We will not change the valuation of contributions, but if we find that other students got more points for "the same thing", we will adapt.
Please do make sure your code runs, and your text renders.
MSc Projects:
Initial Ideas
Previous projects
Next Week: Pontus Stenetorp
Next: More Deep Learning
Open Items Lecture 19
¶
Assignment 1 example solutions:
modelling lower-case/upper case
caching: looking at the most recent histories and build a local LM + interpolate
motivate by repetition in songs
special treatment of
BAR
OOV parameters
cross-validation of parameters to see robustness
motivate higher order models by showing hit-rate
Next: injecting prior knowledge into NNs
In [ ]: