Assignment 1



Question 1. In the car last year, my son (then four years old) generated the following utterance:
You beeped her out of the way so we could wented to school.
Well, yes, ok, I did in fact beep my horn at the car in front of me on our way to school. Children tend to notice these things. But the light was green, and she wasn't moving! For the record, my horn beeping was what we call a "gentle tap", which is officially sanctioned, at least in Idaho.

From an NLP perspective, my son's sentence is chock full of interesting stuff. For each of the following terms or concepts, (a) briefly define what the term means, preferably in your own words, (b) briefly identify where the concept is illustrated in this example, and (c) where relevant, comment on any ways in which this sentence might be particularly challenging for an NLP system.

The point of this question is to make sure that you've at least encountered these concepts, even if you have to look them up in the index of Manning & Schuetze or on Wikipedia. Most or all of them should be familiar to anyone who has taken CompLing1 or equivalent. Don't spend more than 5 minutes on any one of these. And, again, it's ok for people to work together and discuss things, as long as you're telling me explicitly who deserves credit for what. If you're not a native speaker of English, I strongly encourage you to talk with one.


Question 2. Recall that Bayes's Rule is often used in "Bayesian updating", since it is used to take a prior probability distribution and update it on the basis of observed evidence to produce a posterior distribution. Before, after, hence updating; get it?

This notion of "updating" is particularly useful when applied iteratively. Consider a scenario where we have a prior distribution P(H) over hypotheses, and then we observe evidence E1 and use Bayes's Rule to update our distribution to

               P(E1 | H) P(H)
  P(H | E1) =  --------------
                 P(E1)
The distribution P(H|E1) can now be viewed as a new, more informed prior. Therefore if we see a new piece of evidence E2, we can use Bayes's Rule again to derive a new posterior. Since P(H|E1) is the prior now, the result will take the form
                  X  P(H | E1)
  P(H | E1,E2) =  ------------
                       Y

Derive the simplest possible form of the full expression, i.e. fill in X and Y, assuming that E1 and E2 are independent.


Question 3. First, if necessary, review Manning & Schuetze Section 2.1, on foundational concepts in probability theory.

Then, go to Wolfram Alpha and try typing the following things into the box at the top (and then hit Enter or click the "=" sign).

Fun, huh? Ok, now try it with the Beta distribution:

3(a) Look at the PDFs (probability density functions) for each one, and explain in plain English what each of these Beta distributions represents, when interpreted as a Bayesian prior for the probability of heads. (To keep things standard, let's all assume that the prior belief is about the probability of heads for a possibly-unfair coin.) What is your expectation about what the probability of heads will be, and how strongly do you think so?

3(b) Let's take the notion of "expectation" in (3a) a little more literally, as in expected value, also known as the mean of the distribution. Try typing BetaDistribution[α,β] for some other values of α and β, and look at the number Wolfram Alpha reports as the "mean". Can you figure out what the formula for the mean is, as a function of α and β? (If you can't give an exact formula, at least comment on some properties of the mean. If you already knew the formula, it's ok to tell me so. But don't just look up the mean of the Beta distribution and copy it down; what would you learn from that?!.)