Note: I've deliberately made this week's assignment an optional extra credit assignment, since the timing overlaps with Assignment 3. However, I'm making it a lot of extra credit -- 50% of an assignment -- to encourage you to do it even though it's not required.
There are two different mini-projects to choose from. Choose one.
You can do this for the simple case of just bigram probabilities, although it would certainly also be interesting to see what things look like for higher-order ngrams. If you're doing that, I recommend using an existing language modeling module (e.g. in NLTK) or toolkit (e.g. KenLM and SRILM). As usual, just make sure to clearly identify any software that you didn't write yourself in your PDF writeup.
You should do your experiment for English and at least one other language. As one possible source of data to work with, see the NLTK corpora. The Brown corpus would be a smallish but decent source for English (just remove POS tags and lowercase everything; it's already tokenized); you could also augment that with other English corpora, although be careful to make sure that your sources don't overlap, which would throw off your counts. For other languages, one source would be the NLTK samples from the Europarl corpora; if you were feeling more ambitious you could look into the full Europarl corpora, which I believe are quite a bit larger. Or feel free to identify other sources of text.
You are not required to actually succeed in replicating the research results in the paper -- that would take a lot of work, since they used fairly sizeable corpora that you probably don't have easy access to. In fact, you're not even required to compute correlations (though that would be nice); we'll be happy if you produce charts that look roughly like their Figure 2, no error bars required (though that would be nice also). What you do need to do involves briefly summarizing what the study is about in your own words to show that you understand it, clearly describing for us what you did in your own experiment (along with any simplifying choices you made and why/how you made them), and telling us what you you found. If your results are very different from what you would have expected, suggest some explanations why (and ideally support your explanation with data). The more thoughtful and clearly written your writeup the better.
Note that we have not tried this ourselves, so, again, remember that the goal is not to "get it right", but to show you understand the ideas and that you're taking a thoughtful approach. The writeup matters.