In this assignment, you will get some hands-on experience using association scores to identify interesting collocations in a corpus.
You should begin by downloading the convote
dataset v1.1 (9.8 Mb, tar.gz format). There are several different
versions of the data in this archive; we are concerned only with the
data_stage_three directory. For those with limited disk space, a way
to extract just the directory of interest is:
gunzip < convote_v1.1.tar.gz | tar xvf - convote_v1.1/data_stage_three
Each file in the data_stage_three directory contains a speech by a legislator.
The filenames contain information about each speech. In the filename template,
###_@@@@@@_%%%%$$$_PMV:
Here's an example. When I used the Democratic speeches as a corpus, and sorted all non-stopword bigrams by log-likelihood value, and did the same separately for Republicans, the phrase middle class ranked 16th for Democrats and 115th for Republicans; conversely, the phrase law enforcement ranked 28th for Republicans and 141st for Democrats. One could argue that this empirical observation is consistent with at least some characterizations of the priorities of the two political parties -- e.g. see discussions of the role that the phrase middle class had during the 2008 presidential debates, like this one.
Even if you're not particularly familiar with American politics, identify similar contrasts and offer your thoughts on why they might or might not be meaningful.
To continue my example, having discovered that middle class might be an interesting phrase, I could use a chi-square test to show that the difference in the frequency of middle class among Democrats and Republicans is statistically significant. (Note that doing a chi-square test in this way, looking at a contingency table of {Democrat,Republican} x {middle_class, NOT middle_class}, is different from using chi-square to find an association between the word middle and the word class. Both cases involve a 2x2 contingency table, i.e. looking for evidence of an association between some binary variable A and a second variable B, but one case is looking for a lexical association between two words, and the other is looking for an association between a bigram and the party of the speaker using that bigram.)
For full credit, be very explicit about the statistical hypothesis testing you're doing. For example, an explicit argument would (at least) explicitly describe null hypothesis H0, identify the test statistic, identify your choice for alpha (the cutoff for statistical significance), give the p-value, and present an argument for why we should draw the conclusions you want us to draw. (It's fine to use some of the same language I used in class.)
You are welcome to use an on-line calculator or off-the-shelf code to do the statistical calculation if you'd like. Though you'll learn more if you do the calculation by hand at least once. :-) If you're thinking of using the chi-square test, note that there are problems using it when counts are small (see Problems in the chi-square test Wikipedia page). Depending on the counts, the likelihood ratio test (G-test) may be a better choice.
In terms of implementation,
You are welcome to work together. If you do so, turn in separate assignments (since the discussion parts will be different), even if you're turning in the same code, but identify who you worked with.
As specified by the TAs.
Again, I strongly encourage you to talk to each other on the class mailing list. You guys are your own best resource.