About The Course
Information theory is the science of operations on data such as compression, storage, and communication. It is among the few disciplines fortunate to have a precise date of birth: 1948, with the publication of Claude E. Shannon's paper entitled "A Mathematical Theory of Communication".
Our course will explore the basic concepts of Information theory. It is a prerequisite for research in this area, and highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. The intimate acquaintance that we will gain with measures of information and uncertainty - such as mutual information, entropy, and relative entropy - would be invaluable also for students, researchers, and practitioners in fields ranging from neuroscience to machine learning. Also encouraged to enroll are students of statistics and probability, who will gain an appreciation for the interplay between information theory, combinatorics, probability, and statistics.
Prerequisites
A solid first (undergraduate) course in probability, as well as the maturity and motivation to cope with some concepts that may be more abstract than you have previously encountered.
Textbooks
The lectures will be self contained and cover all the material that students are expected to know for the homework (weekly exercises) and the exams (midterm and final). A course reader will also be made available (for free). It contains an outline of the lectures, as well as pointers to references that can be consulted for further reading. These pointers are primarily to the textbooks of Cover & Thomas, Gallager, and Csiszár & Körner.
Instructor
Tsachy Weissman joined the faculty at Stanford University in 2003, where he now holds the STMicroelectronics Chair in the School of Engineering. His research focuses on Information Theory, Statistical Signal Processing, the interplay between them, and their applications. He is inventor of several patents and involved in a number of high-tech companies as member of the technical board. Among his recent awards and honors are an NSF CAREER, a joint Information-Theory/Communication societies best paper award, a Horev fellowship for Leaders in Science and Technology, and a Henry Taub prize for excellence in research. He is on the editorial board of the IEEE Transactions on Information Theory, serving as Associate Editor for Shannon Theory.
Frequently Asked Questions
- When does the class start?
The class will start in March 2012.
- What is the format of the class?
The class will consist of lecture videos, which are broken into small chunks, usually between eight and twelve minutes each. Some of these may contain integrated quiz questions. There will also be standalone quizzes that are not part of video lectures. There will be approximately two hours worth of video content per week.
- Will the text of the lectures be available?
We hope to transcribe the lectures into text to make them more accessible for those not fluent in English. Stay tuned.
- Do I need to watch the lectures live?
No. You can watch the lectures at your leisure.
- Can online students ask questions and/or contact the professor?
Yes, but not directly. There is a Q&A forum in which students rank questions and answers, so that the most important questions and the best answers bubble to the top. Teaching staff will monitor these forums, so that important questions not answered by other students can be addressed.
- Will other Stanford resources be available to online students?
No.
- How much does it cost to take the course?
Nothing: it's free!
- Will I get university credit for taking this course?
No.
Anatomy 
Making Green Buildings 
Information Theory 
Model Thinking 
CS 101 
Machine Learning 
Software as a Service 
Human-Computer Interaction 
Natural Language Processing 
Game Theory 
Probabilistic Graphical Models 
Cryptography 
Design and Analysis of Algorithms I 
Computer Security 
Computer Vision