Covariance: how two quantities move together

16 min · 1 drill

In the textbook: Ch. 9.2 Die Kovarianz von Zufallsgrößen

On a hot day a beach kiosk sells more water and more ice cream. How much two quantities move together is measured by covariance. The spread of their sum depends on it. Later in the chapter the risk of a portfolio depends on it too.

In the exam you get a table that shows, for each pair of values of two quantities, how often that pair occurs. From such a table you compute how the quantities are linked.

Every evening a beach kiosk writes down how many crates of water and how many crates of ice cream it sold that day.

The owner already knows two numbers and is looking for a third:

  • how much the water sales jump around;
  • how much the ice cream sales jump around;
  • how much does the total number of crates per day jump around? That decides how much stock to keep.

The first idea is to add the spreads. But the sales rise and fall together, and the total swings more. How much more is told by one number, the covariance.

Look at three days, each equally likely:

DayWaterIce creamTotal
1213
2437
36511

Spread is measured by the variance, the mean square of the deviation from the mean. Water deviates from its 4 crates by −2, 0 and 2, so its variance is 4+0+43≈2.67. Ice cream also has 2.67. The total deviates from 7 by −4, 0 and 4, and its variance is 16+0+163≈10.67, twice the sum 2.67+2.67. The extra comes from sales moving together: it is twice the covariance. So the covariance here is 2.67.

Check yourself. A task at exam level: the joint distribution of X and Y, that is a table with the probability of each pair of values, is P(X=0,Y=0)=0.35, P(X=0,Y=1)=0.15, P(X=2,Y=0)=0.15, P(X=2,Y=1)=0.35. What is the covariance Cov⁡(X,Y), written σxy?

What the covariance shows

Covariance is built from three ideas of the probability chapter: random variable, expected value and variance. Let us recall them on the same kiosk.

The number of crates of water per day is not known in advance: one day it sells 1 crate, another day 4. A number that depends on chance is called a random variable (“Random variable and its distribution”). Call the water sales X and the ice cream sales Y.

Say 1 crate of water sells on 60 % of days and 4 crates on 40 %. The average over many days is called the expected value:

E(X)=1⋅0.6+4⋅0.4=2.2

It is also written μx. The variance V(X)=σx2 measures the spread: it is the average of the squared distance from the expected value. Here the distances are 1−2.2=−1.2 and 4−2.2=1.8:

V(X)=0.6⋅1.44+0.4⋅3.24=2.16

The root of the variance is called the standard deviation: σx=2.16≈1.47. Tasks also write it as Var⁡(X) (“Expected value and variance: centre and spread”).

Now the link itself. Take the distances of both quantities from their averages, X−μx and Y−μy, and multiply them:

  • on a hot day both are above average, both distances are positive, the product is positive;
  • on a cold day both are below average, both distances are negative, the product is again positive;
  • if one is above average and the other below, the product is negative.

The average of this product is called the covariance (German “Kovarianz”, shortened to Cov in formulas):

σxy=Cov⁡(X,Y)=E((X−μx)(Y−μy))

The figure shows ten days of another, bigger kiosk: this data is only for the picture, after it the article goes back to our kiosk. Each point is the pair (water, ice cream) of one day, the dashed lines are the averages μx=5 and μy=5. Six of the ten points lie bottom left or top right, where the product of the distances is positive. The sign depends on the products themselves, not on the number of points: the four far points give 6 each, the two nearer ones 2 each, and the other four together only −5. So the covariance is positive.

Each point is one day. The dashed lines split the field at the averages; look how many points lie bottom left and top right, where both distances have the same sign0510510XYμx = 5μy = 5both above: product > 0both below: product > 0one above, one below: < 0

If the points lay mostly top left and bottom right, the covariance would be negative: one quantity rises when the other falls. If the points are spread evenly over all four corners, the covariance is close to zero. Two properties follow straight from the definition:

  • the covariance of a quantity with itself is its variance: Cov⁡(X,X)=σx2;
  • the order does not matter: Cov⁡(X,Y)=Cov⁡(Y,X).

Covariance from a table

Back to the beach kiosk. For every pair “how much water, how much ice cream” we know how often it happens. Such a table is called the joint distribution (German “gemeinsame Verteilung”):

Y=0Y=2
X=10.40.2
X=40.20.2

For example, the 0.2 top right means: on 20 % of days the kiosk sells 1 crate of water and 2 crates of ice cream. The row sums give the distribution of X alone: P(X=1)=0.4+0.2=0.6, P(X=4)=0.4. The column sums give the distribution of Y alone: P(Y=0)=0.6, P(Y=2)=0.4. They are called the marginal distributions (German “Randverteilungen”).

Computing the covariance from the definition takes long. Multiply out the brackets in it:

(X−μx)(Y−μy)=X⋅Y−μy⋅X−μx⋅Y+μx⋅μy

The expected value of a sum is the sum of the expected values, and E(X)=μx, E(Y)=μy. The expected values of the four terms are E(X⋅Y), then −μx⋅μy, again −μx⋅μy and +μx⋅μy. Together:

E(X⋅Y)−μx⋅μy−μx⋅μy+μx⋅μy=E(X⋅Y)−μx⋅μy

The minus of μx⋅μy appears twice and the plus once, so one minus is left. This is the shift formula (German “Verschiebungssatz”):

σxy=E(X⋅Y)−E(X)⋅E(Y)

From the row sums: E(X)=1⋅0.6+4⋅0.4=2.2.

From the column sums: E(Y)=0⋅0.6+2⋅0.4=0.8. The spread as for X: V(Y)=0.6⋅0.64+0.4⋅1.44=0.96, σy=0.96≈0.98.

Each cell gives “value of X times value of Y times the probability of the cell”:

1⋅0⋅0.4+1⋅2⋅0.2+4⋅0⋅0.2+4⋅2⋅0.2=0+0.4+0+1.6=2.

σxy=2−2.2⋅0.8=2−1.76=0.24. It is positive: on days with high demand for water, more ice cream sells too.

Independence. Two quantities are independent if the value of one does not change the probabilities of the other. For independent quantities E(X⋅Y)=E(X)⋅E(Y), and the covariance is zero. The converse is false. Let X be −1, 0 or 1 with probability 13 each, and Y=X2:

X−101
Y=X2101
X⋅Y−101

Then E(X)=−1+0+13=0 and E(X⋅Y)=−1+0+13=0, so the covariance is 0−0⋅E(Y)=0, although Y is fixed completely by X.

The variance of a sum

Now we can answer the kiosk owner. The variance of a sum is the average of its squared distance. Write the distances as dx=X−μx and dy=Y−μy; the distance of the sum is then dx+dy. The square multiplies out like (a+b)2=a2+2a⋅b+b2:

(dx+dy)2=dx2+2 dx⋅dy+dy2

The averages of the three terms are σx2, 2σxy and σy2. The doubled product from the school formula is exactly what brings in the covariance:

V(X+Y)=σx2+σy2+2σxy,V(X−Y)=σx2+σy2−2σxy

For X−Y we have −dy in place of dy: the square dy2 stays the same, but the doubled product changes sign. That is why the variances still add in V(X−Y).

At the kiosk V(X)=2.16, V(Y)=0.96 and σxy=0.24. Compare two answers for the owner:

  • with the covariance: V(X+Y)=2.16+0.96+2⋅0.24=3.6;
  • by plain adding: 2.16+0.96=3.12.

Both goods rise together, so the total number of crates swings more than plain adding suggests. If the quantities were independent, the covariance would be 0 and the variances would simply add.

The correlation

The covariance depends on units: computed in euros and in cents, it gives different numbers. It does not show whether the link is strong or weak. The correlation coefficient (German “Korrelationskoeffizient”) divides it by both standard deviations and so always lies between −1 and 1:

ρ=σxyσx⋅σy,σxy=ρ⋅σx⋅σy

ρ=1 means the points lie on one rising line, ρ=−1 on a falling one, and at ρ=0 the covariance is zero. At the kiosk ρ=0.241.47⋅0.98≈0.17: the link is positive but weak.

For another pair of quantities, not the kiosk, take round numbers: σx2=16, σy2=25, σxy=−6. The standard deviations are 4 and 5, so ρ=−64⋅5=−0.3: the quantities move weakly in opposite directions.

Trap. Dividing by the variances: −616⋅25=−0.015. If a correlation is given and the formula needs a covariance, first multiply ρ by both standard deviations; never put ρ itself in.

Covariance from the joint distribution

The shift formula, the variance of a sum and the correlation each came with an example of their own. In the exam task two of them follow as a chain from a single probability table:

Exam-style task

Zwei Filialen einer Handelskette melden täglich die Zahl der Reklamationen X beziehungsweise Y. Die gemeinsame Verteilung ist:

Y=1Y=3X=00.30.2X=20.10.4

Berechnen Sie die Varianz der Gesamtzahl X+Y.

  1. a)2.36
  2. b)1.96
  3. c)7.16
  4. d)2.76
  5. e)1.16

The task gives itself away by a table of joint probabilities, as at the beach kiosk. It asks for the covariance, the correlation or, as here, the variance of the total. The covariance comes from the shift formula and then goes into V(X+Y).

a cell is the probability of a pair: X = 2 and Y = 3 together with probability 0.4

The joint table of complaints: first the cells, then the row and column sums, then the row X = 2 for E(XY)Y = 1Y = 3totalX = 0X = 2total0.30.20.10.4
Step 1 of 5Read the table

A cell of the table is the probability of a pair of values: X=2 and Y=3 occur together with probability 0.4. The four cells add up to 1.

Ask yourself: why can the cells of the row X=0 be skipped?

A copy shop counts printer breakdowns X and customer complaints Y every day. We need the variance of the total number of incidents X+Y.

Y=1Y=2Y=3
X=00.30.150.05
X=10.050.150.3

The rows give 0.5 and 0.5: E(X)=0.5, V(X)=0.5−0.25=0.25.

The columns give 0.35, 0.3, 0.35: E(Y)=0.35+0.6+1.05=2, E(Y2)=0.35+1.2+3.15=4.7, V(Y)=0.7.

Only the row X=1: 1⋅0.05+2⋅0.15+3⋅0.3=1.25.

σxy=1.25−0.5⋅2=0.25, V(X+Y)=0.25+0.7+2⋅0.25=1.45.

The distribution as a list. Two quantities have this joint distribution: P(X=1,Y=0)=0.25, P(X=1,Y=1)=0.25, P(X=3,Y=0)=0.15, P(X=3,Y=1)=0.35. Find Cov⁡(X,Y).

The rows give P(X=1)=0.5 and P(X=3)=0.5, so E(X)=2. The column Y=1 gives 0.25+0.35=0.6, so E(Y)=0.6.

Work out E(X⋅Y) cell by cell and subtract E(X)⋅E(Y).

Your turn1 of 1

Joint distribution of X and Y: P(1,0)=0.05, P(1,2)=0.45, P(3,0)=0.45, P(3,2)=0.05, where P(x,y)=P(X=x,Y=y). What is Cov⁡(X,Y)?

Traps. The task has five options: (a) 2.36, (b) 1.96, (c) 7.16, (d) 2.76, (e) 1.16. The right one is (d). Where the wrong ones come from:

  • Covariance forgotten, option (b), 1.96: only the variances added, 1+0.96, as for independent quantities. Check: here σxy=0.4, not zero.
  • Covariance once, option (a), 2.36: 1+0.96+0.4. The doubled product in the square of the sum gives 2σxy.
  • E(X⋅Y) instead of the covariance, option (c), 7.16: 1+0.96+2⋅2.6. From E(X⋅Y) you still subtract E(X)⋅E(Y).
  • Minus instead of plus, option (e), 1.16: 1+0.96−0.8, that is V(X−Y). The task asks for the total.

This is not on the formula sheet. Remember: σxy=E(X⋅Y)−E(X)⋅E(Y); the row and column sums give the distributions of X alone and of Y alone.

At a glance: covariance, variance of a sum, correlation

The topic in a minute

  • Covariance: the average product of the distances, positive if the quantities tend to rise together. By the shift formula:
σxy=Cov⁡(X,Y)=E(X⋅Y)−E(X)⋅E(Y)
  • Independence: independent ⇒ σxy=0. The converse is false.
  • Sum and difference: in V(X−Y) the variances still add, only the sign of the covariance changes:
V(X±Y)=σx2+σy2±2σxy
  • Correlation: always −1≤ρ≤1; divide by the standard deviations, not by the variances:
ρ=σxyσx⋅σy
  • Covariance from the joint distribution

    A table gives the probability of every pair of values of X and Y; the question asks for the covariance, the correlation or V(X+Y).
    σxy=E(XY)−E(X)⋅E(Y)

    Trap: giving E(X⋅Y) as the covariance or forgetting 2σxy in the variance of the sum.

With factors, as in 2X−3Y, the formula for the sum changes; how exactly is shown in “Combinations aX + bY: how weights enter variance and covariance”.

Cards to memorise

Tap a card to see the formula. All topic cards

Formula

How do you compute σxy from a joint distribution?

Formula

Within what range must a covariance lie?

Now practise it

Covariance: how two quantities move together

Trainer · Covariance and portfolios

Practise
Next: Combinations aX + bY: how weights enter variance and covariance
0/1

Was this article helpful?