Covariance from the joint distribution
A table gives the probability of every pair of values of and ; the question asks for the covariance, the correlation or .Trap: giving as the covariance or forgetting in the variance of the sum.
In the textbook: Ch. 9.2 Die Kovarianz von Zufallsgrößen
On a hot day a beach kiosk sells more water and more ice cream. How much two quantities move together is measured by covariance. The spread of their sum depends on it. Later in the chapter the risk of a portfolio depends on it too.
In the exam you get a table that shows, for each pair of values of two quantities, how often that pair occurs. From such a table you compute how the quantities are linked.
Every evening a beach kiosk writes down how many crates of water and how many crates of ice cream it sold that day.
The owner already knows two numbers and is looking for a third:
The first idea is to add the spreads. But the sales rise and fall together, and the total swings more. How much more is told by one number, the covariance.
Look at three days, each equally likely:
| Day | Water | Ice cream | Total |
|---|---|---|---|
| 1 | 2 | 1 | 3 |
| 2 | 4 | 3 | 7 |
| 3 | 6 | 5 | 11 |
Spread is measured by the variance, the mean square of the deviation from the mean. Water deviates from its 4 crates by , 0 and 2, so its variance is . Ice cream also has 2.67. The total deviates from 7 by , 0 and 4, and its variance is , twice the sum . The extra comes from sales moving together: it is twice the covariance. So the covariance here is 2.67.
Check yourself. A task at exam level: the joint distribution of and , that is a table with the probability of each pair of values, is , , , . What is the covariance , written ?
Covariance is built from three ideas of the probability chapter: random variable, expected value and variance. Let us recall them on the same kiosk.
The number of crates of water per day is not known in advance: one day it sells 1 crate, another day 4. A number that depends on chance is called a random variable (“Random variable and its distribution”). Call the water sales and the ice cream sales .
Say 1 crate of water sells on of days and 4 crates on . The average over many days is called the expected value:
It is also written . The variance measures the spread: it is the average of the squared distance from the expected value. Here the distances are and :
The root of the variance is called the standard deviation: . Tasks also write it as (“Expected value and variance: centre and spread”).
Now the link itself. Take the distances of both quantities from their averages, and , and multiply them:
The average of this product is called the covariance (German “Kovarianz”, shortened to Cov in formulas):
The figure shows ten days of another, bigger kiosk: this data is only for the picture, after it the article goes back to our kiosk. Each point is the pair (water, ice cream) of one day, the dashed lines are the averages and . Six of the ten points lie bottom left or top right, where the product of the distances is positive. The sign depends on the products themselves, not on the number of points: the four far points give 6 each, the two nearer ones 2 each, and the other four together only . So the covariance is positive.
If the points lay mostly top left and bottom right, the covariance would be negative: one quantity rises when the other falls. If the points are spread evenly over all four corners, the covariance is close to zero. Two properties follow straight from the definition:
Back to the beach kiosk. For every pair “how much water, how much ice cream” we know how often it happens. Such a table is called the joint distribution (German “gemeinsame Verteilung”):
For example, the top right means: on of days the kiosk sells 1 crate of water and 2 crates of ice cream. The row sums give the distribution of alone: , . The column sums give the distribution of alone: , . They are called the marginal distributions (German “Randverteilungen”).
Computing the covariance from the definition takes long. Multiply out the brackets in it:
The expected value of a sum is the sum of the expected values, and , . The expected values of the four terms are , then , again and . Together:
The minus of appears twice and the plus once, so one minus is left. This is the shift formula (German “Verschiebungssatz”):
From the row sums: .
From the column sums: . The spread as for : , .
Each cell gives “value of times value of times the probability of the cell”:
.
. It is positive: on days with high demand for water, more ice cream sells too.
Independence. Two quantities are independent if the value of one does not change the probabilities of the other. For independent quantities , and the covariance is zero. The converse is false. Let be , or with probability each, and :
| 1 | 0 | 1 | |
| 0 | 1 |
Then and , so the covariance is , although is fixed completely by .
Now we can answer the kiosk owner. The variance of a sum is the average of its squared distance. Write the distances as and ; the distance of the sum is then . The square multiplies out like :
The averages of the three terms are , and . The doubled product from the school formula is exactly what brings in the covariance:
For we have in place of : the square stays the same, but the doubled product changes sign. That is why the variances still add in .
At the kiosk , and . Compare two answers for the owner:
Both goods rise together, so the total number of crates swings more than plain adding suggests. If the quantities were independent, the covariance would be 0 and the variances would simply add.
The covariance depends on units: computed in euros and in cents, it gives different numbers. It does not show whether the link is strong or weak. The correlation coefficient (German “Korrelationskoeffizient”) divides it by both standard deviations and so always lies between and :
means the points lie on one rising line, on a falling one, and at the covariance is zero. At the kiosk : the link is positive but weak.
For another pair of quantities, not the kiosk, take round numbers: , , . The standard deviations are and , so : the quantities move weakly in opposite directions.
Trap. Dividing by the variances: . If a correlation is given and the formula needs a covariance, first multiply by both standard deviations; never put itself in.
The shift formula, the variance of a sum and the correlation each came with an example of their own. In the exam task two of them follow as a chain from a single probability table:
Zwei Filialen einer Handelskette melden täglich die Zahl der Reklamationen beziehungsweise . Die gemeinsame Verteilung ist:
Berechnen Sie die Varianz der Gesamtzahl .
The task gives itself away by a table of joint probabilities, as at the beach kiosk. It asks for the covariance, the correlation or, as here, the variance of the total. The covariance comes from the shift formula and then goes into .
a cell is the probability of a pair: X = 2 and Y = 3 together with probability 0.4
A cell of the table is the probability of a pair of values: and occur together with probability 0.4. The four cells add up to 1.
Ask yourself: why can the cells of the row be skipped?
A copy shop counts printer breakdowns and customer complaints every day. We need the variance of the total number of incidents .
The rows give and : , .
The columns give , , : , , .
Only the row : .
, .
The distribution as a list. Two quantities have this joint distribution: , , , . Find .
The rows give and , so . The column gives , so .
Work out cell by cell and subtract .
Your turn1 of 1
Joint distribution of and : , , , , where . What is ?
Traps. The task has five options: (a) 2.36, (b) 1.96, (c) 7.16, (d) 2.76, (e) 1.16. The right one is (d). Where the wrong ones come from:
This is not on the formula sheet. Remember: ; the row and column sums give the distributions of alone and of alone.
The topic in a minute
Trap: giving as the covariance or forgetting in the variance of the sum.
With factors, as in , the formula for the sum changes; how exactly is shown in “Combinations aX + bY: how weights enter variance and covariance”.
Tap a card to see the formula. All topic cards
Was this article helpful?