Information theory

Definitions and basic properties of the amount of information provided by the realization of an event and the entropy (in Shannon's sense) of a random variable, simple conditional entropy of a random variable, mutual information of two v.a.
Coding theory, decipherability and ambiguity, spontaneity of codes, Sardinas and Patterson algorithm, Kraft-MCMillan's necessary and sufficient condition for the existence of a decipherable code with words of given lengths
Coding optimality (of v.a.), links between the entropy of a v.a. and the average length of code words. and the average length of the associated code words (Shannon theorems), Huffman algorithm
Transmission of information through noisy memoryless channels, some important channels, channel capacity, calculation of capacity in simple cases, decoding problem, uniform-error-bound decoding scheme, Shannon's fundamental theorem on the possibility of correct transmission with arbitrarily large probability and rate less than capacity (without demonstration).

Discrete probability

1. General information theory
2. Coding theory
3. Optimal codes, Huffman's algorithm
4. Introduction to noisy channels, capacity

authorized documents