How Computers Trawl a Sea of Data for Stock Picks
Funds managing billions hunt for investment clues in newswires, weather, Twitter
Staff work on side projects in Two Sigma's 'Hacker's Lab,' which co-founder David Siegel says helps them think differently about investing.
By BRADLEY HOPE
April 1, 2015
NEW YORK—In SoHo offices where robots occasionally ply the hallways, dozens of Ph.D. scientists with degrees in fields like astrophysics, immunology and linguistics huddle every day around computer screens that show billions of dollars zapping around the world.
Their goal: to give their secretive hedge-fund firm a leg up in investing the $24 billion it has under management. Scientists at the firm, Two Sigma Investments LLC, program its machines to cull torrents of information from sources like newswires, earnings reports, weather bulletins and Twitter.
They then write trading algorithms that make stock picks based on what they call “signals” in those data.
Two Sigma is part of a new frontier in computerized investing, in which scientists and engineers with little formal financial training are trying to funnel massive computing power into predicting securities prices by drawing from clues in news and data.
“The secret of the markets is they can be predicted,” says Alexander Migdal, a former Soviet physicist in Princeton, N.J., who writes algorithms to predict securities prices based on a broad set of data, such as news feeds. “Not 100%, of course, but just enough to make a difference, to make a profit.”
Instead, these practitioners’ aim is to get an advantage over human stock pickers by writing algorithms that are smarter and faster than any person could be in scouring the world’s information, finding patterns and making trading decisions in stocks, bonds, options, futures or currencies.
At Two Sigma, the strategy works like this:
In determining how to trade the stock of, say, a major big-box retailer, Two Sigma’s scientists and mathematicians devise dozens of computer-trading models related to the stock.
One model would automatically pore through analysts’ research for patterns in how they view the retailer—much as a human broker might. Another would look for clues in Twitter: It might identify one pattern—a growing number of customers tweeting complaints, say—and correlate that with another pattern, such as data showing fewer people visiting stores.
Additional algorithms would do other tasks human investors traditionally perform: watching for the stock price to break through a 200-day average, say, or monitoring whether executives are buying or selling company shares.
Each model would produce a trade suggestion, which would go through an algorithm written to weigh each idea based on its model’s historical performance and other factors. Then, a risk-management algorithm would check the suggested trade to make sure it didn’t overexpose the firm to the stock or sector. Finally, an execution system would automatically place a trade.
A model might take weeks for scientists to perfect. Once programmed into computers, a model might act in seconds or less.
“Most investment managers operate more or less the way they operated decades ago: Their investments are made using their brains,” says Two Sigma co-founder David Siegel.The firm’s systems constitute artificial intelligence, he says, and “represent the future of investment management.”
The firm typically holds investments for days, weeks or months.
Investing arms race
Companies like Mr. Siegel’s are beginning a new investing arms race on and off Wall Street. Some are tiny startups. Major quant firms are also exploring the strategy. Hedge-fund firms Renaissance Technologies LLC, with $25 billion under management, and D.E. Shaw & Co., with $36 billion, have explored similar operations, say people familiar with the companies. Both firms decline to comment for this article.
Such firms are part of a broader quest among businesses of all types to profit from so-called big data, the world’s growing flood of information on just about everything, by doing what’s often called “data mining.”
“The computational power available today, combined with the ability to store huge amounts of data, has allowed us to do things that were simply not possible before,” says Petter Kolm, director of the mathematics-in-finance program at New York University’s Courant Institute of Mathematical Sciences, which has sent graduates to Two Sigma and other firms that do data mining.
“We can solve problems now in real-time,” he says, “that would have taken days and weeks a decade ago.”
Firms practicing the approach tend to be secretive, and there are no data on how many companies or how much money is involved. People at such firms, and academics who study them, say strategies they write into their algorithms vary, as do the levels of human involvement during trading.
Some of the strategies worry some who say they may expose investors to undue risk. Among skeptics is Ray Dalio, founder of Bridgewater Associates LP, the world’s biggest hedge-fund firm. While not speaking specifically of Two Sigma, Mr. Dalio believes some such methods risk placing big bets on “spurious” relationships, a Bridgewater spokesman says. Professors from University of California, Davis, and several other institutions warned in an April 2014 research paper of a trend of “overfitting” in math-based trading by hedge funds and other money managers, in which random correlations are interpreted wrongly as strong relationships.
‘Pseudo-math’
They concluded that “pseudo-mathematics” and “financial charlatanism” were running rampant on Wall Street. Such bad math, they wrote, “is a large part of the reason why so many algorithmic and systematic hedge funds do not live up to the elevated expectations generated by their managers.”
Some practitioners of the data-mining approach have failed to make it a business. Flyberry Capital LLC, a $2 million Cambridge, Mass., hedge-fund firm tested trading strategies based on reacting immediately to news events such as earthquakes near the coast of Japan. Its founder said he was especially interested in predicting how different investors would react to events like surprise changes in nonfarm-payroll statistics or the national energy supply. The firm closed earlier this year because it couldn’t attract enough investors, a spokesman says.
Two Sigma casts itself as a safe bet in a jittery, interconnected world where human judgment alone is no longer effective. “Do you want your doctor operating off of gut instinct?” Mr. Siegel asks. “I don’t. I want my doctor to be analytical in their diagnosis and use scientific procedures in their treatment. The same can be said for investment management.”
Mr. Siegel, 53 years old, founded Two Sigma in 2001 with mathematician John Overdeck,45. It was ranked 22nd largest of 305 U.S. firms as of September 2014 by HedgeFund Intelligence, a data-and-research firm.
Last year, it raised a $3.3 billion “macro” fund—one of the largest hedge funds launched since the financial crisis. Another, its $6 billion Compass fund focusing on futures and currencies, returned more than 25% in 2014. The company’s website says its investors “include some of the world’s largest corporate and public pension plans, sovereign wealth funds, research institutions, educational endowments, hospitals and healthcare systems, and foundations.”
Two Sigma’s funds all take a big-data approach. Among its data sources are news bulletins, National Weather Service reports, market data, tweets and information from smartphone users who have agreed to be tracked by a retail-trend-analysis company.
Two Sigma Investments was founded in 2001 by David Siegel, left, and John Overdeck, pictured here at the hedge-fund firm's Manhattan offices. PHOTO: PHILIP MONTGOMERY FOR THE WALL STREET JOURNAL
Employees writing the algorithms typically have backgrounds in fields like physics, chemistry or machine learning—scientists adept at parsing large amounts of data. More than 100 employees have doctorates.
Thirty years ago, it was easier to make investment picks because the world wasn’t as interconnected, Mr. Siegel says. “Here’s the problem: What affects the price of a share of Apple stock? The answer: Pretty much everything. Absolutely every little thing has some effect. Every sale, every earthquake.”
Mr. Migdal, the former Soviet physicist, after leaving academia started a high-frequency-trading firm using computers to zip in and out of markets, earning tiny profits on hundreds of thousands of transactions. But he believes the profit opportunity there is diminishing. His new company, Migdal Research LLC, is devoted to longer-term predictions based on a broader data set.
Mr. Migdal, 69, compares data to water drops before they form a river. “Many little movements of the drop may become a flow,” he says. “Not all the events in the financial world take the form of dramatic and obvious announcements. Quite often, they begin as small drops moving in the same direction in a way that isn’t immediately visible to the human observer.”
Still, says NYU’s Mr. Kolm, computers aren’t close to being omniscient: “For the majority of financial-prediction models, the degree of certainty is much, much weaker” than even weather forecasts.
One risk, critics say, is a repeat of the 2007 “quant meltdown,” when models with similar assumptions broke down simultaneously at many firms. Long-Term Capital Management L.P.’s collapse in the late 1990s is also a stark memory: Backed by two Nobel Prize winners, it used mathematical methods to identify arbitrage opportunities but imploded in 1998, causing ripples across the financial sector. Bridgewater, whose founder has expressed concern about data-mining, says it has invested in capability to analyze data. But its strategies are based on a “deep understanding of the logical cause-effect economic relationships that drive markets,” the spokesman says. In other words, humans come up with hypotheses and test them with powerful computers, rather than letting computers drive the entire process.
Two Sigma recently suffered a large loss when one fund dropped more than 12% in January after the Swiss National Bank abolished a currency cap and the Franc surged, says a person familiar with the fund. The fund was up 8% in February, the person says
.But the firm touts long-term consistency. One of its first funds, Spectrum, has an annualized return of 9.49% after fees and hasn’t had a down year since its 2004 creation, say people familiar with its performance.
Inspiration from robots
A walk through its offices shows how far it is from traditional Wall Street. In a room called the “Hacker’s Lab” where staff work on side projects, a researcher prints out an ergonomic keyboard on a 3-D printer while another teaches a robot to play air hockey. A robot from the lab occasionally plays shuffleboard in hallways.
Mr. Siegel earned a computer-science Ph.D. at Massachusetts Institute of Technology, where he studied at its Artificial Intelligence Laboratory. He is a lover of robotics and believes the Hackers Lab encourages staff to think differently about investing. He calls human investors “non-computer traders.”
To comb through data 24 hours a day, the firm has more than 100 teraflops of power—more than 100 trillion calculations a second—and more than 11 petabytes of storage, the equivalent of five times the data stored in all U.S. academic libraries.
It tightly guards its strategies, but a recent hiring campaign aimed at engineers from firms like Facebook Inc. and Twitter Inc. gives a glimpse. Prospective recruits don virtual-reality headsets that bring them through a frenetic Times Square and fly them over buildings in Dubai. “A lot to take in, isn’t there?” a male voice says. “Shopping, trends, traffic.” What if, the voice continues, there were a burst of headlines about a volcanic eruption in Kamchatka, Russia? If the ash caused air-traffic delays, what would that mean for oil prices? “And that’s what we do at Two Sigma,” the voice says, “look for connections that make sense of it all.”
The company, which has grown to about 750 employees and is looking to hire 100 more engineers, needs brain power for a constant challenge: Where physicists in academia seek immutable laws, scientists writing these trading algorithms unearth patterns whose value disappears quickly as others discover them.
Profits hinge more on whether these firms’ cultures of experimentation can repeatedly give an edge than on discovering a “fortune’s formula,” says James Weatherall, author of “The Physics of Wall Street: A Brief History of Predicting the Unpredictable,” who has studied firms like Two Sigma.
The risk, he says: “If everyone is using the same models, you end up with weird market behavior.”
