📡 Information Theory — Intro#

Scope — How information is quantified, transmitted, and constrained by noise and uncertainty.

Key concepts#

• Information — reduction of uncertainty about a variable or message.
• Entropy — measure of uncertainty or average information content.
• Channel — medium through which information is transmitted, often with noise.

Seed Q&A triads#

• Q: What does information theory study at its core?
  A: How information can be measured, encoded, transmitted, and decoded efficiently and reliably.

• Q: What is entropy in intuitive terms?
  A: A measure of how uncertain or unpredictable a message source is.

• Q: Why is noise important in communication systems?
  A: Noise introduces errors and uncertainty, limiting how accurately information can be transmitted.

Quick activities#

• Compare the entropy of a fair coin toss versus a biased coin and explain the difference.