Scribd
Upload
9 views1 page

Information Theory and Channel Capacity Guide

This document discusses information theory and channel capacity concepts. It contains example problems calculating information content, entropy, Shannon-Fano coding, and channel capacity. Key calculations include: the information content of a picture with 211,000 pixels and 8 brightness levels is 3 bits per pixel; the entropy of a single letter in English is 4.164 bits; and the required SNR for a 3kHz voice line to transmit at 5kbps is 3.37dB.

Uploaded by

Ir Muhamad
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
9 views1 page

Information Theory and Channel Capacity Guide

This document discusses information theory and channel capacity concepts. It contains example problems calculating information content, entropy, Shannon-Fano coding, and channel capacity. Key calculations include: the information content of a picture with 211,000 pixels and 8 brightness levels is 3 bits per pixel; the entropy of a single letter in English is 4.164 bits; and the required SNR for a 3kHz voice line to transmit at 5kbps is 3.37dB.

Uploaded by

Ir Muhamad
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd

Tutorial : Information and Channel Capacity

1. Which of the following are uniquely decipherable codes? a. 001, 0110, 1100, 00011, 00110 b. 0, 010, 01, 10 c. 0, 100, 101, 11

[yes,no,yes]

2. A TV picture contains 211,000 picture elements. Suppose that this can be considered a digital
signal with eight brightness levels for each elements. Assume that each brightness level is equally probable a. What is the information content of a picture? [3bits] b. What is the information transmission rate (bits per second) if 30 separate pictures(frames) are transmitted per second [18.99Mbits/s] 3. The following lists shows the probability of occurance of each letter of the alphabet in a standard English text A=0.081 G=0.016 S=0.066 Y=0.019 B=0.016 H=0.051 M=0.022 R=0.060 T=0.096 Z=0.001 C=0.032 I=0.072 N=0.072 U=0.031 D=0.037 J=0.001 O=0.079 V=0.009 E=0.124 K=0.005 P=0.023 W=0.020 F=0.023 L=0.040 Q=0.002 X=0.002

a. Find the entropy associated with sending a single letter [4.164 bits/letter] b. What is the information content of the three-letter word THE? Assume the letters are c.
4. independent of each other. [12.5bits] You are told that the probability of occurance of the word THE is 0.027. Use this information to find the information content of the word THE. [5.2bits]

Find the minimum average length of a code with five messages having probabilities; a. 1/8,1/8,1/8,1/8,1/2 [2 bit/code] b. 0.2,0.2,0.3,0.15,0.15 [2.27 bit/code] Derive the Shannon-Fano coding for above cases

5. You are given six messages with the following probablities;


M1=0.1 M2=0.25 M3=0.2 M4=0.25 M5=0.1 M6=0.1
[2.46 bit/msg]

a. Find the entropy of this code


b. c.

Use Shannon-Fano coding to design avariable-length code that is efficient Find the average length per code word for answer part b.

6. Find the channel capacity C if the SNR is 6dB and a standard broadcast AM channel is used (5kHz bandwidth).
[11.58kHz, 13.9MHz]

Repeat for a broadcast TV channel (6-MHz bandwidth), and compare the answers.

7. A voice-grade telephone line has a bandwidth of 3kHz. If you wish to transmit at a rate of 5kbps,
what SNR is required?
[3.37 dB]

You might also like