From prices to weights

1Notation and conventions

The basket has n securities observed over T trading days. All vectors are columns; the prime denotes transposition and 1 is the vector of ones.

The trading year is 252 days. Annualisation is arithmetic — mean times 252, standard deviation times the square root of 252 — which is the convention of mean–variance theory, where mean and variance are the moments being optimised. It is not a compound growth rate and is not meant to be.

2Data and sample construction

2.1Source and raw series

Prices are daily closes from Yahoo Finance, requested with dividend and split adjustment on, from 1 January 2000. No other data enters the estimates: the catalogue's reference data — asset class, country, sector — serves only the structural indices of section 9, never the optimisation.

History stops at the end of yesterday: every row dated today is excluded, whatever the exchange and whatever the hour. An intraday price is not a close, but the reason for a single cut-off is another one: a boundary that moves during the day — at every exchange close, and at every expiry of the price cache — makes two identical analyses requested an hour apart come out with different weights. The boundary is midnight Greenwich, the only instant after which a day is over on every market, including crypto and currencies, which never close. It costs one session of freshness and in exchange makes the analysis reproducible: the same securities, on the same day, give the same numbers. The period actually used is always stated on the page.

2.2Cleaning

A null or negative price sends the return to infinity, and from there infinity propagates into mean and covariance until the matrix is meaningless with nothing to signal it. Prices at or below zero are therefore masked before any calculation, and their count is reported in the data panel.

2.3Shared calendar

Securities listed on different exchanges do not share dates: Milan closes on 26 December, New York does not. The union of dates contains every single exchange's holidays; the exact intersection, with fifty securities, empties out — if each misses 5% of sessions, 0.95⁵⁰ ≈ 7% of the calendar survives.

The basket's calendar is defined on the EXCHANGES, not on the securities, and for each exchange it is the exchange itself that declares it. Let ℬ be the set of the m venues the basket is listed on and 𝒮_b the set of sessions of venue b, taken from its official calendar: t is a session of the basket when it is a session of at least a share q_c = 0.8 of the venues.

A venue's sessions are a fact about the venue, and are to be asked of it: on 4 July New York is closed and Milan is not, on Easter Monday it is the other way round, and neither depends on which securities one chose to analyse. They come from exchange_calendars, the reference library for exchange calendars. Deducing them from the securities present instead — «the venue was open if 80% of its securities traded» — meant that a security added to the basket, possibly discarded by the optimiser, could take sessions away from everyone else: eight sessions the newcomer had not traded brought EVERYONE's observations down from 6,697 to 6,689, and the means moved. For a venue whose calendar is not known, counting securities returns, and it is the only case left in which the basket touches the calendar.

Cells still empty inside the calendar are filled by carrying forward the last known quote, for no more than 5 days. What remains uncovered after that is a real suspension: that date drops out, because the basket is not observable all together.

Returns are simple and computed on the resulting calendar:

A return that jumps a gap of more than 15 calendar days is discarded: it would be a two-week move lined up with one-day moves.

2.4Securities excluded, and why

Two rules remove securities from the sample, and in both cases the page states which one and why.

• A security with its own prices on fewer than 50% of the calendar's dates brings carried-forward values, not prices: its estimated volatility comes out lower than the truth and the minimum-variance portfolio piles up on it.
• A security whose removal would double the usable history is on its own costing more than all the others together. The doubling threshold is not fine tuning: it separates «this security costs a piece of history» from «this security costs the history».

Below 30 common observations nothing is estimated: there is not even enough material to start.

3Moment estimation

3.1Why not the sample covariance

The sample covariance is badly conditioned when observations are not many more than assets: the smallest eigenvalues are systematically underestimated, the optimiser mistakes them for near-riskless directions and piles the portfolio onto them. This is why the sample frontier is notoriously unstable and weights jump from day to day.

3.2Softened correlations

Let X be the T × n matrix of returns and X_c the same, column-centred. From the unbiased sample covariance one extracts the standard deviations and the correlation matrix:

Correlations are pulled toward zero by a share δ that depends only on how much history there is — not on the number of securities:

and the covariance used downstream is reassembled with the sample standard deviations, which remain each security's own:

3.3Positive definiteness

Rounding can leave slightly negative eigenvalues, and then w′Σw may come out negative with a square root of a negative number downstream. The matrix is symmetrised and eigenvalues are clipped from below at λ_max · 10⁻¹². The resulting condition number is reported in the diagnostics.

3.4Histories of different length

The common sample is the most prudent choice and the most wasteful: the window starts at the first quote of the youngest security, so an ETF born in 2021 throws away fifteen years of everyone else's history.

Each cell is therefore estimated on the securities that cell is about: a security's mean on ITS OWN series, all of it, and a pair's correlation on the days both were there. No young security shortens anything for anyone any more and — this is the property that matters — adding a security to the analysis cannot change any of the others' figures, because none of those figures ever looked at it.

The two classic objections to this choice are known, and must be faced. First: figures estimated on different samples may compose a matrix that is not positive semidefinite, and the optimiser would immediately find the negative-variance directions, because they are exactly what it looks for. The defence is softening correlations toward zero (§3.2), which lifts the eigenvalues, and adding — if the smallest is still negative after that — the MINIMAL correction that brings it above zero. With overlaps differing by a few days out of thousands it is almost never needed.

The second objection is that means become incomparable with one another: a security listed two years ago carries the mean of its two years, and if those two years were a bull market its mean is high for a reason that is not its own — and the optimiser, which drinks means whole, piles onto it. This is the damage Stambaugh's method (1997) repairs, by regressing the young security on the longer-history securities IN THE BASKET, and it is precisely that «in the basket» that is unacceptable here: adding a full-history security to a basket containing one listed later moved the latter's mean by 0.013 points a year, and with it the last digit of the figures on the page.

The reference is therefore one the basket cannot touch: the INDEX OF THE MARKET the security is listed on — the S&P 500 for New York, the Euro Stoxx 50 for the euro area, the Nikkei for Tokyo. Each security decomposes as r_i = α_i + β_i B_i + e_i: a piece of market and a piece of its own. The first can be extended, because the index exists over the WHOLE period of the analysis, including the years before the security was listed; the second cannot, and stays measured where the security was there. Hence mean, variance and covariance:

mean_i = α_i + β_i · mean(B_i over the whole period)  ·  var_i = β_i² var(B_i over the whole) + var(e_i)  ·  cov_ij = β_i β_j cov(B_i, B_j over the whole) + cov(e_i, e_j on the days of both)

No price is invented: it is only said that of that security one knows how much it moves with its market and how much it swung on its own, and that the market has been known for twenty-five years. A security present over the whole period is left untouched — «the index over the whole» and «the index over its period» are the same thing, and the least-squares decomposition makes β²var(B) + var(e) exactly its sample variance: the extension is worth zero. No threshold between «short» and «long» history is therefore needed, and there are no two nearly identical securities treated differently because one sits a month above a line. Measured: a security listed in 2023, among securities with twenty-five years of history, goes from 16.9% to 23.0% annual volatility — that of its peers — because it inherits the crises of its market that it did not live through, and its expected return falls from 28.5% to 20.7%.

Every piece of the computation — the beta, the index moments, the overlap dates — depends only on that security and its index; the residual covariance only on the pair. Adding a security to the analysis changes none of those quantities. Below 60 observations in common with the index the beta is noise and nothing is extended; the beta is clipped at ±3, beyond which it describes not a security but an error in the data; if the index does not arrive from the source, that market goes unextended and the analysis carries on. The results page names the securities that were extended.

Stambaugh's method (1997) remains implemented and is switched on by an environment variable, for whoever prefers the finer estimate to stability. With securities ordered by history length, on the long block L one estimates μ_L and Σ_LL using all its observations; over the overlap the short block S is regressed on the long one,

and the short block's moments are rebuilt using the long block's full moments:

Centring on the overlap mean in (7) is what makes (8) a real correction: the young security's mean is carried to the long period through its beta, that is, corrected for its listing window having been a bull or a bear market.

The result's positive semidefiniteness is algebraic, not numerical. The joint matrix factorises as a congruence of a block diagonal:

As long as Σ_LL and Σ_ε are positive semidefinite, so is the result.

3.5Regularising the regression (optional method)

With a hundred long securities and two hundred days of overlap, ordinary least squares is rank deficient: it would fit noise and the betas would be worthless. The regression therefore runs on the first k principal components of Σ_LL, orthogonal by construction, with

that is, at least ten overlap observations per component. The coefficients then return to security space as B = β′V′, where V holds the components in columns. With ample overlap and few components requested the result coincides with ordinary least squares.

Below 20 days of overlap no regression is run at all: a wrong beta is worse than one security fewer, because it propagates through its whole covariance row. Those securities stay out and the page says so.

3.6Annualisation

4Risk-free rate

Sharpe is excess return divided by volatility, so the rate is no detail: it is half the numerator. It must also be the rate of the currency the return is measured in — subtracting SOFR from a euro-quoted portfolio is not an approximation, it is a unit error.

4.1Sources and convention

The official overnight rates are used: €STR published by the European Central Bank for the euro, SOFR published by the New York Federal Reserve for the dollar. Both are quoted ACT/360 and are converted to a calendar year before entering the Sharpe ratio:

Without that step the rate would stay 1.4% too low. The value used, the source and the reference day are printed next to each portfolio's Sharpe: whoever compares two numbers must be able to see whether they were measured with the same yardstick.

4.2Multi-currency baskets

When the reference currency is chosen by the analyst, that currency's rate is used and nothing else: the risk-free rate is what keeping money idle earns, and the money is kept by the analyst, in their own currency.

For internal calls, where nobody has chosen, the basket is already measured in a composite unit — returns are not converted at the exchange rate — and the risk-free counterpart of that unit is the average of the same currencies' rates. The correct weights would be the portfolio's, but those are known only after optimising while optimising needs the rate: the universe's shares are used instead — how many securities per currency — and the part not covered by a primary source is reweighted onto the covered part.

where π_v is the share of securities quoted in currency v and 𝒞 the set of currencies with a primary source configured. If no source answers, the rate is zero and the page says so: Sharpe goes back to being the ratio of return to swing.

5Portfolio metrics

5.1Return, risk, Sharpe

5.2Value at risk

VaR is historical, not parametric: the series of the portfolio's daily returns is built and its percentile taken. For confidence level c,

computed at levels c ∈ {0.90, 0.95, 0.99}. The three levels do not repeat one another: 90 says what an ordinary bad day looks like, 99 what a really bad one looks like, and the distance between them says how heavy the tail is — whether the portfolio loses slowly or in jolts.

6The two portfolios

6.1Feasible set

Long only and fully invested. The minimum weights written by the analyst enter as a lower bound on each variable:

No short selling, no leverage, no per-security cap. The set is a convex compact polytope, non-empty as long as 1′m ≤ 1.

6.2Minimum variance

Variance is minimised, not volatility: the square root does not move the minimiser but worsens the gradient's conditioning near zero. The problem is a convex QP and is solved with SLSQP from two starting points — equal weights and inverse variance — keeping the better one.

6.3Tangency portfolio

Maximising the Sharpe ratio directly is a non-convex problem: a solver started from a single point can stop at a local maximum, and the «optimal» point ends up off the frontier. The Cornuéjols–Tütüncü reformulation is used instead. With excess e = μ − r_f1, if max_i e_i > 0 one solves

and normalises:

Problem (19) is convex, hence has a unique global minimum, and (20) returns exactly the tangency portfolio: the Sharpe ratio is scale-invariant, which is what makes the final normalisation legitimate.

When no security beats the risk-free rate the tangency does not exist — no line from r_f touches the frontier from above — and the page shows the minimum-variance portfolio, declaring the case.

6.4Minimums in the auxiliary variable

In (19) the optimisation is over y, not weights, and a lower bound on w is not a lower bound on y: it depends on 1′y, itself a variable. The condition can be rewritten in y, however, and comes out linear — so the problem stays a convex QP:

6.5Constraints exact by construction

A numerical solver respects bounds within its tolerance, and «within tolerance» is not «always»: a weight can come out at 9.9998% where the minimum asked was 10%. On a constraint the analyst wrote, that is not acceptable. Every feasible portfolio can be written in exactly one way,

and the correspondence is one-to-one. The solver's solution is therefore mapped onto this parametrisation: take the part exceeding the minimums, rescale it to sum one, and rebuild w. The result respects the minimums to the digit, not to the tolerance.

6.6Choosing among candidates

Besides the QP solution, two direct maximisations from different starting points are computed as a safety net. Among the candidates the highest Sharpe ratio rounded to six digits wins and, on a tie, the highest return: same price in swing, more result. Without the tolerance two Sharpes would never be equal — they differ in the fifteenth digit — and the second criterion would never come into play.

7Efficient frontier

7.1The parametric problem

For each target return a convex QP is solved:

with R_k equally spaced between the minimum-variance portfolio's return and the extreme corner's. Unconstrained, the corner is the single best security; with minimums one can no longer put everything on one, and it becomes «all that is left over, on the best one».

The efficient branch uses 70 points upwards; the dominated branch uses 40 downwards, replacing arg max with arg min in the choice of corner. Together the two branches are the boundary of the feasible region, the textbook bullet shape.

7.2Refinement order

Points are not solved left to right but by successive bisection: first the two ends, then the midpoint, then the midpoints of the two halves, and so on. Two useful consequences. First: every new point sits between two already solved, so it starts from the interpolation of their solutions — along the frontier weights move almost linearly, and the solver arrives nearly converged. Second: if time runs out, what is lost is the curve's density, not its extent.

The time check happens before starting a point, not after: the cost is estimated as the average of the points already solved, and the next one is not started unless there is room for two. The first 17 points are solved regardless — below that threshold there is no curve, only a polyline.

8Cloud of random portfolios

The chart's background is a sample of feasible portfolios drawn at random. A uniform Dirichlet would look like the obvious choice, but with n assets the weights concentrate around 1/n: the cloud becomes a central blob that never comes near the frontier, making it look detached from the data.

Sampling therefore uses a mixture of symmetric Dirichlets with varying concentration:

with probabilities (0.25, 0.30, 0.25, 0.20). Small α produce concentrated portfolios near the simplex's corners; large α produce spread ones. Together they genuinely cover the feasible region.

Draws are 40,000 up to thirty securities and 20,000 beyond, with a fixed seed — the same analysis always returns the same picture — and are computed in blocks of 4,000 scenarios, so memory does not depend on the number of draws.

In the drawing each scenario is a dot, coloured by its own Sharpe ratio on the viridis scale. Not all of them reach the image: one dot every k is drawn, with k fixed and not random, up to twelve thousand dots. The sample is therefore regular — same shape and same relative density as the full set — and the memory the drawing needs does not grow with the number of draws.

9The three structural indices

Alongside the two portfolios the page shows three numbers from 0 to 100 that contain no return and no risk. All three answer the same question posed at different levels — how much the portfolio rests on a single thing — and are called independence, concentration and variety.

9.1The shared curve

Two of the three indices start from an effective count and turn it into a score with the same function, so that «54» means the same thing on both cards:

rounded to an integer and clipped to [0, 100]. The curve saturates: 1 effective count is worth 0, 2 are worth 28, 3 are worth 49, 5 are worth 74, 8 are worth 90. Going from eight to ten moves little, and rightly so — the real difference lies between one and three.

9.2Independence

How many distinct securities one is actually holding. From the correlation matrix derived from the covariance already estimated,

one takes the eigenvalues λ₁, …, λ_n and counts the effective number of independent directions as the spectrum's participation ratio:

The equality on the right holds because the trace of a correlation matrix is n. The value is clipped to [1, n] and the score is v(N_ind). Five securities all correlated at 0.8 give 1.4: five names, one single bet.

The card also shows the average correlation over all pairs and the most alike pair, which are the same C read differently — no extra measure.

9.3Concentration

How much the portfolio depends on a few securities. On the optimal allocation w one computes the inverse of the Herfindahl index:

and it is the number of securities that, all equally weighted, would give the same concentration: ten at 10% give ten, one at 90% with nine residual ones gives about one. The score is again v(N_eff).

Independence and concentration can give opposite answers, and both are needed: ten identical securities are barely independent but hardly concentrated; two uncorrelated securities are independent but very concentrated. The card's big number runs with the score — higher means weight more spread — while the word beside it runs the other way, because weight poorly spread is high concentration.

9.4Variety

How many different categories the optimal portfolio rests on. It is seen through three lenses — kind of investment, region, sector — taken from the catalogue's reference data and, for funds, from the instrument's name.

For each lens, with q₁, …, q_K the shares normalised on the classified part, the effective number of categories is again an inverse Herfindahl:

The two resulting scales differ, however, and deliberately. Kind has a known denominator — the families are three: equities, bonds, commodities — so the score can say «one out of three» instead of «far from the theoretical maximum»:

Region and sector have no fixed total — the world's regions are five, the sectors eleven — and use a saturating curve calibrated on how many categories really exist, resting on a non-zero floor:

A lens enters the score only if it covers at least 50% of the portfolio: below that it knows too little, and counting it badly is worse than not counting it. The percentages shown on screen, by contrast, always refer to the part the question applies to — «what sector are you?» asked of a gold bar is not an unanswered question, it is a question that does not arise.

9.5Lens weights and single-asset-class portfolios

In the ordinary case the asset class carries three quarters of the score and geography one quarter: it is the asset class that decides what kind of risk the portfolio depends on. Sector is shown but carries no weight — it is the noisiest lens, and it repeats what concentration already says better.

An all-equity basket, however, has by construction a single asset class, and with the ordinary weights the main lens would score 33 and drag three quarters of the judgement with it. But that variety exists — not at the asset-class level, one floor down, in sectors and regions. Writing

as the degree of «single class», the weights slide continuously between the two settings:

The raw score is the weighted average of the lenses that actually carry weight, and two corrections act on it.

The equity correction. The category count is blind to direction: 80% equities and 20% bonds has the same variety as its opposite, but the latter is a portfolio without an engine. With a the equity share of the classified part,

The single-class penalty. Moving the weight is not absolution: an all-equity portfolio, however well spread across sectors, depends on one single thing — when the market falls, it all falls together.

and the final score is

The ceiling for a single-class portfolio remains 80% of that of one with genuinely more than one, and now two things say so instead of a lone multiplier: kind pinned at 33 inside the weighted average, and (35) for the rest. Indeed 0.20 · 33 + 0.80 · 100 = 86.6 and 86.6 · 0.92 = 80.

When no lens passes the coverage threshold, variety is not judged: the card stays without a score rather than showing an invented one.

10BlissPoint Score

The arithmetic mean of the available indices, rounded:

The three lenses weigh the same: there is no reason to say one counts more than the others, and an arbitrary weighting would be harder to explain than to write. Variety can be missing — a portfolio of multi-asset funds has neither sector nor country for half its weight — and then the average is taken over the two remaining, declaring how many lenses it rests on. Below two nothing is computed: an average of one is not an average, it is that score under another name.

The page also shows which of the three is the lowest, because that is the information the average erases and almost always the one you want to know next.

11Reading bands

The bands exist to convey whether a number is a lot or a little, and nothing else. They are not a standard and not a forecast: they depend on the period and the data analysed, and the same portfolio can change band if the historical window changes.

The zero threshold is the only non-arbitrary one of the four on Sharpe, because it flips the sign of the meaning: below it, the portfolio returned less than the risk-free rate and the swing endured was not paid for. Above 2 the ratio is so good that it almost always depends on a lucky historical window rather than a stable property — which is why the highest band carries a warning rather than a compliment.

12Parameters

Every constant that enters the calculations, with the value in force. They are fixed in code and identical for every analysis: two different baskets can be compared because they were measured with the same yardstick.

13Declared limits

What follows is not a list of flaws to be fixed one day: these are the conditions under which the figures in this document hold. Whoever uses them must know them.

13.1It is all history

Mean and covariance are estimates on past prices. Past return does not predict future return, no figure shown is a forecast and no percentage is a promise. In particular, the mean is by far the most fragile input of mean–variance theory: small estimation errors amplify in the weights. That is why the page also shows the minimum-variance portfolio, which does not depend on the mean at all.

13.2No currency conversion

Returns are price changes in each security's listing currency, without conversion. A mixed basket is therefore measured in a composite unit. Conversion was implemented and withdrawn: correct in theory, it required exchange-rate series, and every missing rate was one more excluded security on a source that already struggles to serve prices — in exchange it moved the frontier with no way to tell whether the move was the exchange rate or a missing datum.

13.3A delisted security still shortens the window

Histories that begin at different times are no longer a problem: each security brings its own, and the means of younger ones are straightened against their market index (§3.4). The right-hand side is another matter: a security that stopped trading before the others still shortens the window for everyone. A publication delay within the five carry-forward days is forgiven — otherwise, at six in the evening, a Milan security would take the last session away from a New York one — but a real ending stays an ending, and is paid for in common.

13.4Classification is not perfect

The three lenses of §9.4 rest on the catalogue's reference data and, for funds, on what the name declares. A fund whose name does not say what it holds stays unclassified, and the page writes it out with name and weight instead of forcing it into a box. The distinction between «missing datum» and «question that does not arise» is explicit: gold has no sector, and saying «unclassified» would be saying something false.

13.5VaR is in-sample

It is a percentile of the returns observed over the same window the portfolio was optimised on. It is not an out-of-sample estimate and says nothing about tails that did not show up in that window.

13.6No leverage, no shorting, no costs

The feasible set (17) excludes short selling and leverage. Fees, bid–ask spreads, taxes and rebalancing costs do not enter the calculation: the weights are a theoretical destination, not an order to be executed.

14References

• Markowitz, H. (1952). «Portfolio Selection». The Journal of Finance, 7(1), 77–91.
• Sharpe, W. F. (1994). «The Sharpe Ratio». The Journal of Portfolio Management, 21(1), 49–58.
• Ledoit, O. and Wolf, M. (2004). «A well-conditioned estimator for large-dimensional covariance matrices». Journal of Multivariate Analysis, 88(2), 365–411.
• Stambaugh, R. F. (1997). «Analyzing investments whose histories differ in length». Journal of Financial Economics, 45(3), 285–331.
• Cornuéjols, G. and Tütüncü, R. (2007). Optimization Methods in Finance. Cambridge University Press — convex reformulation of the tangency portfolio.
• Meucci, A. (2009). «Managing Diversification». Risk, 22(5), 74–79 — effective number of independent bets.
• Hirschman, A. O. (1964). «The Paternity of an Index». The American Economic Review, 54(5), 761 — the index used in eq. (28) and (29).
• Kraft, D. (1988). A software package for sequential quadratic programming. DFVLR-FB 88-28 — the SLSQP algorithm used for (18), (19) and (23).

15Revisions to the method

Listed here are the changes that alter a result or the way it is obtained. Presentation-only changes do not appear: they move no number.

15.1Version 1.1 — the currency is declared by the source

• Quotation currency. It used to be inferred from the symbol suffix, then from the internal catalogue. Both are guesses, and in London they are wrong ones: the same venue lists lines in dollars, in pounds and in pence. It now comes from the metadata of the very response that carries the prices, and is stored. A wrong currency meant applying another currency's exchange rate to a security for its whole history, hence invented return and volatility.
• Prices in minor units. London quotes several lines in pence and the source says so («GBp»). The factor of one hundred cancels in returns but not in a market value: quantities inferred from an amount came out a hundred times too small. Prices in minor units are now brought back to the main unit on entry.
• Currency risk. It is included by default: prices are converted into the reference currency before any estimation, so the exchange rate enters the mean, the volatility and above all the correlations. A switch excludes it and looks at securities in their quotation currency.
• Minimum weight constraints by category. Besides a minimum on a single security, a minimum share can be imposed by asset type, geography or sector. It enters as a linear inequality on the weights, with a feasibility check before the optimisation and an exact repair after, as already done for per-security minima.
• BlissPoint Score. It is no longer the average of the three structural indices. It is the performance grade — a monotone function of the Sharpe ratio, hence comparable across analyses — corrected by at most one fifth by the portfolio's structure. See §10.
• Composition, three axes instead of one. The basket's variety is no longer a single index: it is read separately by asset type, geography and sector. Commodities and crypto count as global exposure, not as a missing area.
• Stability. Given the same prices and risk-free rate, two analyses of the same basket return identical numbers. The quantities that depend on machine load — the density of the cloud in §8 and the number of points on the curve in §7 — affect the drawing only and enter no reported figure.