Assignment 3


This assignment is due as usual, at the start of next class. However, since I am giving you less time, I have scoped down the assignment and this one will be worth 50% of a regular homework assignment.

First, a couple of notes to follow up on things I mentioned in class, as promised, although they're not needed for this assignment.

For the mathematically inclined and/or those who liked the idea of deriving an information measure from a set of properties such a measure should have, here are several derivations of the definition of entropy based on axioms about such properties. The one I mentioned in class is from A.I. Khinchin, Mathematical Foundations of Information Theory, Dover, New York, 1957.

Also, in case you're curious, here's the pointer to an algorithm for creating optimal codes using Huffman coding (see also this interesting discussion by Huffman's nephew).


There are two different problems to choose from. Choose one.