Information Theory Core

Master Entropy and Mutual Information

What you’ll learn
Understand and differentiate modes of convergence – including convergence everywhere, almost everywhere, in mean square (MS) and in probability.
Apply the law of large numbers – both Bernoulli’s and Borel’s strong law — to analyze the long-term behavior of random sequences.
Explain and compute information-theoretic measures such as entropy, joint entropy, conditional entropy, and apply the chain rule in problem-solving contexts.
Work with theoretical tools and proofs – — making use of lemmas, worked examples, and illustrative problems to strengthen mathematical reasoning.

Requirements
Information Theory Fundamentals

Description
This course, Information Theory Core, is the natural sequel to Information Theory Fundamentals, also by the same instructor. While the earlier course built the essential foundation of information theory through probability concepts, this course moves decisively into the heart of information theory.We begin by consolidating probabilistic tools with lectures on convergence in mean square, convergence in probability, distributional convergence, and the law of large numbers—cornerstones that prepare us for the rigorous study of information. With this background in place, we introduce entropy, the central quantity in information theory. You will learn its meaning and its properties.The course then explores the non-negativity of entropy, the change of base property, and step-by-step worked examples to build deep intuition. These early insights serve as gateways to advanced topics such as coding theorems, data compression, and the probabilistic underpinnings of communication systems, which will be addressed in subsequent lectures.By the end of Information Theory Core, you will not only be comfortable manipulating definitions and rules but will also appreciate their conceptual power. This course is designed for learners in mathematics, computer science, electrical engineering, and related fields who are serious about mastering information theory as both a theoretical and applied discipline.

Who this course is for
Seniors and graduate students looking to enter into information theory proper

https://www.udemy.com/course/information-theory-core/