Covariance and Correlation: From Statistical Foundations to Portfolio Diversification
Updated: Oct 20
Introduction
The risk of a portfolio is not the sum of the risks of its parts. Two assets that each swing 20% a year can combine into a portfolio that swings 20%, 14% or almost nothing, depending entirely on how they move together. The covariance and correlation are the measures of that joint movement, and they sit at the centre of portfolio construction, hedging, VaR, and stress testing.
The covariance tells you the direction of co-movement in the units of the data. The correlation rescales it to a pure number between -1 and +1 that can be compared across any pair of assets. Together with standard deviation, they are the full input set for the classical theory of diversification that Harry Markowitz formalised in 1952.
The formulas are short. The hard part is that correlations are estimated with large error, change over time, rise exactly when diversification is needed, and describe only linear relationships. This article builds the measures from first principles, then spends most of its time on how they behave in real portfolios and where they mislead.
Practical Challenges
5 problems make correlation one of the least reliable inputs in a risk model, despite being one of the most important.
Scale: Covariance carries the units of both variables, so a covariance of 15,000 is not obviously "bigger" than one of 25. Comparisons need correlation, but portfolio arithmetic needs covariance; practitioners constantly move between the two.
Instability: Correlations shift with regimes. The correlation between US equities and Treasuries was broadly positive before 2000, mostly negative for the two decades after, and turned positive again in 2022 as inflation drove both down together. A hedge calibrated in one regime can add risk in the next.
Crisis Convergence: Correlations among risk assets tend to rise sharply in sell-offs, as investors de-risk across the board and common factors dominate. Diversification measured in calm markets overstates the protection available in a crash.
Estimation Error: A 3-year monthly correlation rests on 36 observations. Its standard error is roughly (1 - ρ²) / √n, so a true correlation of 0.3 is estimated within about ±0.15 only one time in three. A 500-asset covariance matrix has 1,24,750 distinct correlations to estimate, often from fewer observations than assets.
Linearity and Consistency: Pearson correlation captures only linear dependence and says nothing about whether assets crash together. And a correlation matrix built pair by pair, or edited by hand for a stress scenario, may not be internally consistent: it can imply negative portfolio variance, which breaks optimisers, VaR engines and simulations.
What's Covered
The article moves from definition, to calculation, to portfolio use, to the ways correlation fails.
Covariance – the direction of co-movement and why its size is hard to read.
Correlation – standardising covariance and why it is bounded.
Reading correlation values – strength bands, r², and what zero does not mean.
Example – two stocks, five months, step by step.
Correlation and diversification – portfolio variance, minimum-variance weights, beta and hedge ratios.
Covariance matrices in practice – positive semi-definiteness, shrinkage and PCA.
Time-varying correlation – rolling and EWMA estimates, crisis behaviour and sign flips.
Estimation pitfalls – price levels versus returns, asynchronous data, significance and causation.
Correlation in risk management and regulation – stress testing, correlation risk and FRTB correlation scenarios.
Limitations and alternatives – rank correlation, tail dependence and copulas.
Best practices and common errors.
Appendix – derivations of the bounds, invariance, minimum-variance weights and hedge ratios.
It builds on the companion pieces on return definitions and standard deviation.
Covariance: The Direction of Co-movement
The covariance measures whether two variables tend to be above or below their means at the same time. For a population of N paired observations:
and for a sample of n observations, with Bessel's correction:
How the Sign Arises? Each term multiplies two deviations. Both above the mean, or both below, gives a positive product; one above and one below gives a negative product. The sum is positive when same-side days dominate, negative when opposite-side days dominate, and near zero when neither does.
X relative to Mean | Y relative to Mean | Product | Pushes Covariance |
Above | Above | + | Up |
Below | Below | + | Up |
Above | Below | - | Down |
Below | Above | - | Down |
Useful Properties:
Cov(X, X) = Var(X): variance is the covariance of a variable with itself.
Covariance is symmetric: Cov(X, Y) = Cov(Y, X).
It is bilinear: Cov(aX + b, cY + d) = ac Cov(X, Y). Adding constants changes nothing; rescaling changes the covariance by the product of the scale factors.
Why Its Size Is Hard to Read: That last property is the problem. Measure returns in percent instead of decimals and every covariance rises 10,000-fold; the relationship has not changed at all. Covariance is therefore excellent for arithmetic (portfolio variance is built from it) and poor for interpretation. Its natural units here are %², or decimal² if returns are in decimals.
Correlation: Standardising Covariance
Correlation divides covariance by both standard deviations, cancelling the units and leaving a pure number between -1 and +1:
This is the Pearson product-moment correlation, formalised by Karl Pearson in the 1890s, building on Francis Galton's work on regression. In sample form, use sXY, sX and sY. The n - 1 factors cancel, so sample and population correlations computed on the same data are identical; Excel's CORREL and PEARSON give the same answer.
What Does The Scaling Mean? Correlation is the covariance of the two variables after each is standardised to mean zero and standard deviation one. It asks: relative to how much each variable moves on its own, how much do they move together?
Why Is It Bounded? By the Cauchy-Schwarz inequality, the absolute covariance can never exceed the product of the standard deviations, so |ρ| ≤ 1. Equality holds only when one variable is an exact linear function of the other (appendix A1).
Invariance: Correlation is unchanged by any positive linear transformation of either variable. Returns in percent or decimals, prices in dollars or euros at a fixed rate, temperatures in Celsius or Fahrenheit: the correlation is the same. A negative scale factor flips the sign. This invariance is what makes correlations comparable across assets, currencies and asset classes.
Moving Between the Two: Portfolio and risk calculations need covariances; people interpret correlations. The conversion runs both ways:
This is how most risk systems store the problem: a correlation matrix plus a vector of volatilities, each updated on its own schedule, combined into a covariance matrix when needed.
Reading Correlation Values
Correlation measures how tightly two variables cluster around a straight line, not how often they move in the same direction and not how much one moves when the other does.
Strength Bands: There is no universal standard, but these conventional bands are widely used. The sign gives direction; the absolute value gives strength.
|ρ| | Label | r² (variance shared) | Typical Financial Example |
1.0 | Perfect | 100% | Same asset in two currencies at a fixed rate |
0.7 to 1.0 | Strong | 49% to 100% | Two large banks; adjacent points on a yield curve |
0.3 to 0.7 | Moderate | 9% to 49% | Two equity sectors; equities and high-yield credit |
0 to 0.3 | Weak | 0% to 9% | Equities and gold, in many periods |
0 | None (linear) | 0% | — |
Use r², Not r, To Judge "How Much": The square of the correlation is the share of one variable's variance explained by a linear fit on the other. A correlation of 0.7 sounds high but explains under half the variance. A correlation of 0.3 explains 9%. This is why a hedge with 0.7 correlation still leaves substantial residual risk (section 5).
Correlation Is Not Slope: Two assets with correlation 0.9 can have very different move sizes: if one is three times as volatile, it moves roughly three times as far. The slope of the relationship is beta, ρ × σY / σX, which combines correlation with relative volatility.
Zero Correlation Is Not Independence: If X is symmetric around zero, X and X² have zero correlation despite one being an exact function of the other. In markets, an option's PnL and its underlying's return can show low linear correlation while being tightly, non-linearly linked. Independence implies zero correlation; the reverse holds only in special cases such as jointly normal variables.
Lower Is Better for Diversification, All Else Equal: There is no "sweet spot" at moderate correlation; every reduction in correlation reduces portfolio variance for given weights and volatilities. What limits the search for low-correlation assets is their expected return, liquidity, and the stability of the correlation itself.
Example: 2 Stocks, 5 Months
5 months of returns for a technology stock (A) and a utility (B) give a sample covariance of 1.90 %² and a correlation of 0.87.
Step 1: Means:
A: (2.5 - 1.2 + 3.1 + 0.8 - 0.5) / 5 = 0.94%.
B: (1.8 - 0.8 + 2.2 + 0.6 + 1.1) / 5 = 0.98%.
Step 2: Deviations, Products, and Squares:
Month | A Return | B Return | A Deviation | B Deviation | Product (%²) | A Dev² (%²) | B Dev² (%²) |
1 | 2.5% | 1.8% | +1.56 | +0.82 | +1.2792 | 2.4336 | 0.6724 |
2 | -1.2% | -0.8% | -2.14 | -1.78 | +3.8092 | 4.5796 | 3.1684 |
3 | 3.1% | 2.2% | +2.16 | +1.22 | +2.6352 | 4.6656 | 1.4884 |
4 | 0.8% | 0.6% | -0.14 | -0.38 | +0.0532 | 0.0196 | 0.1444 |
5 | -0.5% | 1.1% | -1.44 | +0.12 | -0.1728 | 2.0736 | 0.0144 |
Sum | 0.00 | 0.00 | 7.6040 | 13.7720 | 5.4880 |
Step 3: Covariance: sAB = 7.6040 / (5 - 1) = 1.90 %². 4 of 5 months contribute positively; only month 5, when A fell and B rose, pulls the other way.
Step 4: Standard Deviations:
sA = √(13.7720 / 4) = √3.443 = 1.86%.
sB = √(5.4880 / 4) = √1.372 = 1.17%.
Step 5: Correlation: ρ = 1.90 / (1.86 x 1.17) = 0.87.
Reading It:
The relationship is strong and positive, but r² = 0.77 means about three-quarters of the variance of one is linearly explained by the other, and a quarter is not.
A correlation of 0.87 does not mean the stocks move together 87% of the time. Here they moved in the same direction relative to their means in 4 of 5 months; that is a different statistic.
The beta of A on B is 1.90 / 1.372 ≈ 1.39: when B is 1 point above its mean, A tends to be about 1.4 points above its own.
With five observations, this estimate is almost meaningless statistically. The t-test in section 8 gives t ≈ 3.1 on 3 degrees of freedom, just short of the 3.18 needed for significance at 5%: a correlation of 0.87 cannot be distinguished from zero on this sample. Real analysis needs far more data.
Correlation and Diversification
Correlation is the term in portfolio variance that turns two risky assets into something less risky than either. For two assets:
Equal Volatilities: A 50/50 portfolio of two assets with 20% volatility each:
ρ | Portfolio Variance | Portfolio Volatility | Reduction vs. 20% |
+1.0 | 0.0400 | 20.0% | 0% |
+0.5 | 0.0300 | 17.3% | 13% |
0.0 | 0.0200 | 14.1% | 29% |
-0.5 | 0.0100 | 10.0% | 50% |
-1.0 | 0.0000 | 0.0% | 100% |
Unequal Volatilities: Diversification can beat the safest asset. Take asset 1 at 20% volatility and asset 2 at 10%, with ρ = 0.3. The weight in asset 1 that minimises portfolio variance is:
The resulting portfolio has volatility of 9.8%, below the 10% of the less risky asset on its own. Adding a small amount of a riskier, imperfectly correlated asset lowers risk. This holds whenever ρ is below σ2 / σ1 (here 0.5).
Many Assets: With N assets, the cross-terms multiply. Three assets already need three correlations; in matrix form, with weights w and covariance matrix Σ:
As N grows, the variance terms shrink in importance, and the covariance terms dominate. Portfolio risk converges to the average covariance, which is why average correlation, not the number of holdings, sets the floor on diversified risk.
Beta: An asset's beta to the market is its covariance with the market divided by the market's variance, or equivalently ρ × σ_asset / σ_market. It is the slope of the asset's returns on the market's, and the measure of systematic risk in the CAPM.
Hedge Ratios and Hedge Effectiveness: To hedge a position S with an instrument F, the minimum-variance hedge ratio is h* = ρ σS / σF, the same slope as beta. The fraction of variance removed is ρ², and the remaining volatility is σS √(1 - ρ²).
Hedge Correlation | Variance Removed | Volatility Remaining |
0.70 | 49% | 71% |
0.90 | 81% | 44% |
0.95 | 90% | 31% |
0.99 | 98% | 14% |
A 0.9-correlated proxy hedge still leaves 44% of the original volatility: basis risk is larger than correlation numbers make it sound.
Covariance matrices in practice
For N assets, the covariance matrix Σ is N x N and symmetric, with variances on the diagonal and covariances off it. It has N(N + 1)/2 distinct entries: 55 for 10 assets, 500, 500 for 1,000.
In Excel – With returns in a T x N range R (one column per asset):
Mean: = AVERAGE(column) ..for each asset, giving a 1 x N row M
Covariance Matrix: = MMULT(TRANSPOSE(R - M), R - M) / (ROWS(R) - 1)
Volatility: = SQRT(diagonal of the covariance matrix), as an N x 1 column S
Correlation Matrix: = Cov / (S * TRANSPOSE(S))
In older Excel versions, these are array formulas entered with Ctrl+Shift+Enter. For single pairs, COVARIANCE.S gives the sample covariance (COVARIANCE.P the population version) and CORREL the correlation, which is the same either way. In Python, numpy's cov and corrcoef, or pandas' df.cov and df.corr, do the same in one line.
Positive Semi-Definiteness: A valid covariance or correlation matrix must give non-negative variance for every possible portfolio: w'Σw ≥ 0 for all w, equivalently all eigenvalues ≥ 0. A matrix estimated from one complete, synchronised dataset satisfies this automatically. Matrices built pair by pair from different date ranges, with missing data handled pairwise, or edited by hand for a stress scenario, often do not.
Correlations constrain each other. If ρAB = 0.9 and ρBC = 0.9, then ρAC must be at least 0.62:
A stress scenario that sets ρ_AC = -0.5 alongside the other two produces a matrix with a negative eigenvalue: some portfolio would have negative variance, Cholesky decomposition fails, and Monte Carlo simulation cannot run. The standard repair is to find the nearest valid correlation matrix, by eigenvalue clipping or Higham's (2002) alternating projections algorithm, and to report how far the repaired matrix moved from the intended one.
Estimation Noise and Shrinkage: With N assets and T observations, the sample matrix is noisy when T is not much larger than N, and singular (not invertible) when T ≤ N. Optimisers amplify that noise, piling into assets whose correlations happen to be underestimated. Shrinkage estimators, such as Ledoit-Wolf, blend the sample matrix with a structured target (constant correlation, or a factor model) to trade a little bias for much lower error.
Factor Structure and PCA: Eigen-decomposition of the matrix reveals how many independent drivers really matter. For a government yield curve, the first three principal components (level, slope, and curvature) typically explain the large majority of daily variance. Factor and PCA-based models replace thousands of pairwise correlations with a handful of factor exposures, which is both more stable and easier to stress.
Time-Varying Correlation
A single correlation over a long sample averages across regimes that may look nothing alike. Three estimators track how it changes.
Rolling Windows: Recompute the correlation over a moving window: for example, 36 monthly returns updated each month, or 60 or 250 daily returns updated each day. Plotting the series shows regime shifts, crisis spikes, and how persistent changes are. The window length trades responsiveness against noise, and a single extreme observation affects the estimate for exactly one window length, then drops out abruptly.
EWMA: Apply the same exponential weighting used for volatility to covariances, then divide by the EWMA volatilities:
Using the same λ for every variance and covariance guarantees a valid (positive semi-definite) matrix; mixing decay factors does not.
DCC-GARCH. Engle's dynamic conditional correlation model fits GARCH volatilities to each asset, then models the correlations of the standardised residuals with their own mean-reverting dynamics. It is the standard econometric tool for forecasting correlation.
What the Data Shows:
Crisis Convergence: Correlations among equities, credit, and other risk assets rose sharply in 2008 and in March 2020, as forced selling and a single dominant factor (risk appetite) overwhelmed asset-specific drivers. Safe-haven assets such as government bonds often moved the other way.
Asymmetry: Equity correlations are typically higher in falling markets than in rising ones, so diversification is weakest in exactly the scenarios investors fear most.
Sign Changes: The stock-bond correlation is the clearest example of a relationship that reverses: broadly positive when inflation is the main risk (as before 2000 and again in 2022), negative when growth is the main risk.
A Caution in Reading Crisis Correlations: Correlations measured in high-volatility periods are mechanically higher even if the underlying relationship has not changed, because higher common-factor volatility raises measured correlation (Forbes and Rigobon, 2002). For risk management, the distinction matters less, since the portfolio suffers either way, but it matters when deciding whether "contagion" has occurred.
Estimation Pitfalls
Most misleading correlations come from what was fed in, not from the formula.
Correlating Prices Instead of Returns: Price levels trend, and any two trending series show high correlation in levels whether or not they are related. Two independent random walks routinely produce level correlations above 0.8. Always compute correlation on returns or changes, never on prices, index levels, or yields (use yield changes).
Asynchronous Data: Markets close at different times. A shock during the US session hits Asian prices the next day, so same-day correlations between India and New York understate the true relationship. Correlations also tend to fall as sampling frequency rises (the Epps effect), because prices react to common news with small lags. Use synchronised snapshots, lagged terms, or weekly returns.
Stale Prices and Illiquidity: Assets that do not trade every day show artificially low correlation with liquid markets, for the same reason they show artificially low volatility.
Outliers: One extreme joint observation can swing a correlation by tenths on a short sample. Check influential points; a single crisis day may be the most informative observation, or a data error.
Statistical Significance: Under the null hypothesis of zero correlation, with roughly normal data:
For confidence intervals, use Fisher's transformation z = atanh(r), which is approximately normal with standard error 1 / √(n - 3). A sample correlation of 0.5 from 60 monthly returns has a 95% interval of roughly 0.28 to 0.67: "the correlation is 0.5" really means "somewhere between weak-moderate and fairly strong".
Data Mining: In a 100-asset universe, there are 4,950 pairs. Even if every true correlation were zero, about 250 of them would appear significant at the 5% level by chance. A striking correlation found by scanning many pairs needs an economic reason and an out-of-sample check.
Correlation Is Not Causation: Ice-cream sales and drownings correlate because both rise in hot weather. In markets, two assets often correlate because both load on a common factor (rates, the dollar, risk appetite), not because one drives the other. The distinction matters for hedging: a hedge that relies on a shared factor works only as long as that factor dominates both.
Correlation in Risk Management and Regulation
Every aggregate risk number embeds a correlation assumption, explicit or implicit. Knowing where it sits is the first step to stressing it.
VaR and Diversification Benefit: Parametric VaR aggregates positions through the covariance matrix; historical simulation embeds whatever co-movement occurred in the lookback window. Either way, the gap between the sum of standalone desk VaRs and the firm-wide VaR is the diversification benefit, and it is entirely a correlation estimate. It is also the part of the number most likely to shrink in a crisis.
Stress Testing: Correlation enters stress tests in two ways. Factor-based scenarios shock rates, spreads, equities, and FX jointly according to a narrative or a historical episode, so co-movement is built into the scenario and is internally consistent. Correlation stresses instead override the matrix itself, for example by setting equity correlations to 0.9. The second approach must respect positive semi-definiteness (section 6), and should be reported alongside the repaired matrix so users know what was actually tested.
Correlation Risk: Some positions are exposures to correlation itself: basket and spread options, dispersion trades, quanto products, and credit portfolio tranches. Their PnL moves when correlation moves even if volatilities do not. The 2005 credit correlation dislocation and the failures of Gaussian-copula pricing of structured credit in 2007-08 are the standard reminders that a single correlation parameter can carry large, poorly understood risk.
FRTB Standardised Approach: Within the sensitivities-based method, weighted sensitivities are aggregated within and across buckets using prescribed correlations. Capital is computed under three correlation scenarios: the prescribed (medium) values, a high scenario that scales them up by 25% (capped at 100%), and a low scenario that takes the larger of 2ρ - 100% and 75% of ρ. The charge is the highest of the three, so banks are capitalised against the possibility that their correlations are wrong in either direction.
FRTB Internal Models: Under the expected shortfall approach, diversification across broad risk classes is only partly recognised: the capital measure averages an unconstrained ES with the sum of risk-class ES figures computed separately, with a 50% weight on each.
Credit Capital: The Basel IRB formula for credit risk uses an asset correlation parameter in a one-factor model to translate individual default probabilities into portfolio tail losses; higher correlation means losses cluster and capital rises.
Asset-Liability Management: Pension funds and insurers care about correlation between assets and liabilities, not just among assets. A bond portfolio whose duration matches the liabilities is "risky" in isolation but highly correlated with the liabilities, and so reduces surplus risk.
Limitations and Alternatives
Pearson correlation is a complete description of dependence only for jointly normal (more generally, elliptical) variables. Financial returns are neither, so several complements are used.
Rank Correlation: Spearman's ρ is the Pearson correlation of the ranks; Kendall's τ counts concordant versus discordant pairs. Both capture any monotonic relationship, not just linear ones, are robust to outliers, and are unchanged by any increasing transformation of the data. A large gap between Pearson and Spearman on the same data is itself a warning of outliers or non-linearity.
Tail Dependence: What matters for risk is often whether assets crash together. The lower tail dependence coefficient measures the probability that one asset is in its extreme lower tail given that the other is:
Two jointly normal variables have zero tail dependence for any correlation below 1: extreme joint losses become vanishingly rare. Real markets do not behave that way.
Copulas: A copula separates each asset's own distribution from the dependence structure linking them. The Gaussian copula inherits zero tail dependence; the Student-t copula has positive, symmetric tail dependence; Clayton concentrates dependence in the lower tail. Choosing a copula is choosing how likely joint extremes are, which is why the Gaussian copula's use in structured credit proved so costly.
Conditional (Exceedance) Correlation: Compute correlation separately on days when both assets fall more than a threshold, or when the market is in its worst decile. This directly measures "diversification when it matters", with the volatility-conditioning caveat from section 7.
Measure | Captures | Use When |
Pearson ρ | Linear co-movement | Portfolio variance, parametric VaR, hedge ratios |
Spearman ρ, Kendall τ | Any monotonic relationship | Outliers, non-linear but monotonic links, copula calibration |
Tail Dependence | Joint extremes | Crash risk, stress scenario design |
Copula | Full dependence structure | Monte Carlo, credit portfolios, multi-asset simulation |
Exceedance Correlation | Co-movement in bad states | Testing whether diversification survives sell-offs |
Best Practices and Common Errors
Best Practices
Compute correlations on returns or changes, from one synchronised, complete dataset.
State the window, frequency and weighting with every correlation, and show a rolling history, not just a point estimate.
Attach uncertainty: a Fisher-z interval or at least the sample size.
Check every matrix for positive semi-definiteness before it reaches an optimiser, VaR engine or simulator; repair it with a documented method and report the size of the repair.
Prefer factor models or shrinkage over raw sample matrices when assets are many and history is short.
Stress correlations explicitly, especially upward for risk assets and toward sign changes for hedges; do not rely on calm-period estimates for crisis protection.
Report a tail- or rank-based measure alongside Pearson when positions are non-linear or returns are heavily fat-tailed.
Common Errors
Correlating price levels and finding spurious relationships between trending series.
Reading ρ = 0.8 as "moves together 80% of the time" or as "80% of the risk is shared" (the latter is r² = 64%).
Treating correlation as slope and sizing a hedge on correlation alone, ignoring relative volatility.
Assuming zero correlation means no relationship, or independence.
Building a correlation matrix pair by pair from mismatched histories, then wondering why the optimiser produces extreme weights.
Counting on diversification benefits estimated in quiet markets to hold in a sell-off.
Quoting a correlation from a handful of observations, as in a five-month example, as if it were reliable.
Appendix: Derivations
A1. Why -1 ≤ ρ ≤ 1: Standardise both variables: ZX = (X - μX) / σX and ZY likewise, each with variance 1 and covariance ρ. Variance cannot be negative, so:
Equality requires ZX ± ZY to be constant, meaning one variable is an exact linear function of the other. This is the Cauchy-Schwarz inequality in statistical form.
A2. Invariance to Linear Transformation: For constants a, b, c, d with a, c ≠ 0:
A3. Minimum-Variance Hedge Ratio: Hedging position S with h units of F gives variance:
Setting the derivative with respect to h to zero gives h* = ρ σS / σF. Substituting back, the minimum variance is σS²(1 - ρ²), so the hedge removes a fraction ρ² of the variance.
A4. Two-Asset Minimum-Variance Weight: With w2 = 1 - w1, differentiate the two-asset portfolio variance with respect to w1 and set it to zero:
w1* is positive exactly when ρ < σ2 / σ1, which is the condition for the riskier asset to reduce the risk of a portfolio otherwise held entirely in the safer one.
A5. Consistency Bound for Three Correlations: A 3 x 3 correlation matrix is positive semi-definite only if its determinant is non-negative:
Solving this quadratic in ρAC gives the interval used in Section 6:
Comments