Burton Malkiel on why his classic investment book, 'A Random Walk Down Wall Street,' is relevant 50 years later

This week, Princeton professor Burton Malkiel has published the updated, 50th anniversary edition of A Random Walk Down Wall Street: the Best Investment Guide that Money Can Buy.  More than any other book, it popularized the idea of indexing as an investment strategy and why you can't beat the market. Malkiel was a close friend of Vanguard founder Jack Bogle and spent 28 years on the board of Vanguard. 

Malkiel will be on CNBC's "The Exchange" today at 1:30 PM ET. Below are excerpts from a series of email interviews.  They have been lightly edited for style and clarity. 

You recommended index funds 50 years ago even before index funds existed. Do you still believe that and what is the evidence?

I believe even more strongly than ever that index investing is an optimal strategy and that index funds should constitute the core of everyone's portfolio. Standard & Poor's publishes annual reports showing how actively managed funds compare with index funds. Each year about two-thirds of active managers underperform an index fund. And the one third who outperform in one year tend not to be the same as the one-third who outperform in the next. When you measure performance over a decade or more, 90 percent of active managers are outperformed by a broad-based index fund. On average, active managers underperform the market by about one percent per year. 

You famously said stocks tend to follow a "Random Walk."  What is a Random Walk? 

Random Walk means that the history of past stock market prices cannot be used to predict the future. Sometimes there is some momentum in the market, but momentum strategies do not work reliably, and prices change randomly. Sometimes value stocks and small-cap stocks outperform, but they can underperform for years. So-called "factor funds" have underperformed the market over the past 15 years. 

Despite the evidence, it seems most people don't want to believe the evidence. Why can't we accept that stocks follow a Random Walk?

People fool themselves when they see apparent patterns because streaks are more memorable than randomness. Even sports figures fool themselves into believing that the streak exists and that they have a "hot hand." For example, behavioral psychologists have examined the basketball free throw records of college and pro players who believe that if they make a number of free throws in a row, they are more likely to have success in the next shot. The evidence is quite the opposite. A 50% free throw shooter has a 50% chance of success on the next shot no matter how many previous shots went in. 

You are a big backer of the Efficient Market Hypothesis (EMH), which says that asset prices reflect all available information. But you have pointed out this does not mean that prices are always accurate. Define EMH, what it says, and what it does not say.

Efficient markets do not imply that prices are always correct. Even if everyone valued stocks as the present value of future cash flows, the future can only be estimated. So prices are often "wrong." EMH says that no one knows for sure if they are too high or too low. EMH admits that bubbles can exist but no one knows for sure how much they inflate before popping. Alan Greenspan suggested that the market was "irrationally exuberant" in 1996.  The bubble popped in 2000. Meme stocks like Gamestop sold at bubble levels but hedge fund Melvin Capital went bankrupt shorting them. The market may not be perfectly efficient and it may make egregious errors.  BUT IT IS EXTREMELY HARD TO BEAT. 

Exchange Traded Funds (ETFs), most of which are tied to index funds, are continuing to rake in money.  How do you feel about ETFs? 

I like ETFs if they are broad-based index funds. I do not favor the very specialized ones that really represent active management. I believe that the leveraged ones (such as three times the up or down movement of the market) are really gambling contracts, not investment products. 

Last time we spoke you said the next revolution after index funds is in investment advice. Most advisers charge 1% or more. When it's done online it is 0.25%.  Are there any signs that the "advice revolution" is happening?

I think a revolution in investment advising services is underway, just as indexing itself was a revolution.  Investment advisors charge between 1 and 3 percent per year to manage portfolios for individuals. Software companies can do it effectively for 25 basis points for those who accept fully electronic management and 50 basis points for a hybrid model where you can also talk to a human being from time to time. I work with Wealthfront (a fully electronic) and Rebalance (a hybrid) investment manager. The electronic or computer managers can also provide direct indexing where the fund can efficiently harvest tax losses while maintaining a pre-tax exposure to the broad market. 

A couple years ago you wrote an op-ed in the Wall Street Journal very critical of ESG (Environmental, Social and Governance) funds, saying they were a "self-defeating strategy." Since then, they have come under even more scrutiny. Do you still feel that way?  Why?

ESG funds promise that you can do good for society with your investments and do well financially at the same time. But there is no agreement over what is a "good" investment. Is a natural gas company good because it is the cleanest burning carbon and the bridge we will need for decades on the road to a carbon-free world? Or is it bad because it is a pollutant? Is a munitions maker good because it is providing Ukraine with defensive weapons or bad because its products kill people? Are Meta and Visa good because they are not major polluters, or are they bad because they can cause extreme anguish for teenagers and because they charge exorbitant interest rates to poor people? ESG funds also have high fees and they have been underperforming standard index funds. 

You have always preached of the benefits of a diversified portfolio.  What should a diversified portfolio look like?

Portfolios should be broadly diversified but will be different for people in different circumstances.Young people should dollar cost average by investing regularly and almost exclusively in equity index funds (60/40 stocks to bond for young people, is not appropriate). An investor in his/her 70s and 80s taking required minimum distributions needs a larger proportion of limited duration fixed income. 

The S&P was down almost 20% last year. In years when it has been down 20%, it typically bounces back the following year. What can we expect for stocks in 2023?

I do not make short-term stock market predictions. No one can do this correctly with any consistency. But cyclically-adjusted price-earnings multiples (CAPEs) give the best forecasts for long-run equity returns.  Today CAPEs are well above average. This suggests that returns over the next decade are likely to be below the 9%-10% long-run historical stock market returns. Investors then need to be modest in their expectations and consider that returns could be only half the historical average.