Rise of the Algos - Knight Capital and the Inevitable Catastrophic HFT Triggered Market Meltdown In Our Future

It happened again on Wednesday.  A High Frequency Trading  (HFT) proprietary algorithm crafted by highly compensated brains-on-sticks and nurtured in the bowels of an investment bank escaped into the wild. From its carefully co-located massively parallel high speed trading computer it emerged to wreak havoc on the financial markets.  In this case, it was a happy ending. The algo turned on its master and sucked the very life blood out of the firm that deployed it.

The incident is yet another lights-flashing, klaxon-blaring warning signal that competing HFT algorithms operating with the speed, size and complexity that are commonplace today cannot be adequately regulated, tested, or managed. Their interactions with each other cannot even be understood  or predicted.

Wednesday 8-1-12: The Knight Capital Algo Clusterfrack

John Carney at CNBC:  Latest Market Glitch Shows 'Trading Out of Control'

"Wednesday morning's stock snafu had a familiar ring to it — mysterious volume in trades that simply could not have been made by a human comes surging out of nowhere, causing brief but acute market mayhem By now, many players on trading floors have gotten used to the disruptions that can come from the highly automated new world of high-frequency trading. But that doesn't mean they like it. "This algorithmic trading is kind of out of control," Phil Silverman, managing partner at Kingsview Capital, said as officials at the New York Stock Exchange tried to make sense of what happened... Authorities involved in reviewing the matter said Knight Capital, a trading outfit that employs algorithms used in high-frequency trading, said it experienced "technology issues" with its market-making procedures."

Once Knight Capital's (KCG) algo was identifed as the culprit, the public company's stock was slashed by 1/3 - a $300 million market cap haircut decapitation. And that is before we knew how much the algo lost on the trades. [UPDATE: That was Wednesday's fallout. As I write this Thursday morning KCG's stock has been slashed in half again and the trading loss is being reported at $440 Million]

Tyler Durden at Zero Hedge: This Is What Happens When An HFT Algo Goes Totally Berserk And Serves Knight Capital With The Bill

"We also all know that one should buy low and sell high. At least that is what human traders are taught, and that is what they attempt. Because if one consistently does the opposite, one will simply run out of money. Well, the opposite is precisely what the berserk algo in Knight's Market Making group may have done if Nanex, which has done a forensic analysis of one of the trades in question, is correct. In other words, instead of at least attempting to provide liquidity via limit trades, Knight's algorithm acted as a market order... gone horribly wrong. As the third chart below shows what the algo did with furious repetition and steadfast consistency was to buy at the offer, and sell at the bid, in other words buy high and sell low. Over and over and over and over. As Nanex laconically notes, "In the case of EXC, that means losing about 15 cents on every pair of trades. Do that 40 times a second, 2400 times a minute, and you now have a system that's very efficient at burning money."

 HFT SNAFUS Are Becoming Commonplace With Increasing Financial Damage

Wednesday's event provided an opportunity for investment professionals, financial news networks, and market pundits to recount previous HFT failures.  Once again they can wag their collective  fingers about the complexity, lack of safeguards and unknown dangers of these high speed transactional engines.

Paul Vigna at Wall Street Journal: When Robots Strike: From the Flash Crash to Today’s Snafu

"Today’s trading irregularities, or error, or snafu, or whatever cutesy term this one ends up getting tagged with, is not the first mistake of its kind in the computerized, high-frequency world of the stock market. The 2010 flash crash is the most notorious example, but it isn’t the only one, and indeed, these kinds of errors seem like they may just become an irritating but regular fact of life for traders and investors. Here are some of the most notorious examples of computer-driven snafus. Welcome to the new stock market... Flash Crash.... BATS IPO... Facebook IPO... Sawtooth Trade...."

This is the problem. These "glitches", "SNAFUs", "hiccups", "errors" and "oops"  are happening with sufficient frequency that they are now being treated as unfortunate but normal aspects of today's markets. They are not.

These failures are warning signs that the high frequency nuclear fire burning in the furnace of the financial reactor is spinning out of control,  the containment  has been breached,  and unless the transactional chain reaction is cooled or shut down a catastrophic meltdown is inevitable. But apparently nobody wants to believe that. Instead we see the regulators and executive leadership on Wall Street tapping on the flashing red indicators in the financial control rooms and saying "Do you think this thing is really working?"

Criminal Algos Are a Big Problem But Not The Biggest Problem

Forget that competing algos executing hundreds of thousands of high speed financial transactions every second are indistinguishable from the illegal practice of Quote Stuffing and Front Running. Forget that if the identical transaction patterns were being placed by individuals at speeds that humans could comprehend we'd be seeing perp walks, indictments and convictions. All of this has been known for years but no one is enforcing the law when algos are committing the crime. Is this unconscionable malfeasance on the part of  private and public regulators and compliance officers? Yes. Nevertheless, the consequences of quasi-criminal HFT trading amounts to mouse nuts when compared to the potential financial damage that these algos could and probably will ultimately inflict on the market. The big problem lives at the intersection and interaction of competing HFT programs and trading systems.

There is a popular aphorism on Wall Street "Price is truth". These algos are designed to secure a pricing advantage over other algos and human traders by setting, quoting, offering, cancelling and transacting price quotes. It does not matter how much testing on each individual algorithm or program is done within the vacuum of a market simulation.  It is inherently impossible to predict how the algo will behave in every circumstance when competing with many other algos that are gaming the system for the same price advantage.  And it all happens so fast and and at such high volume, that by the time humans can intervene and pull the plug, the damage is done.

Undoubtedly there is a PHD dissertation and possibly a Nobel Prize to be earned by a Chaos Theory scholar proving that:  1) Complex dueling pricing algorithms will always yield unpredictable results  and 2)  It is mathematically impossible to eliminate the possibility of these systems unintentionally precipitating a market destroying "black swan" event.

A Slow Motion Example of Dueling Pricing Algorithms Gone Awry

An appreciation of the kind of unintended consequences that can result when pricing algorithms compete can be seen in a much simpler example that occurred over a much slower period of time. Consider The Case of the $23.6 Million Dollar Book On Amazon:

"...how about this: $1.7 to $2.1 million for a book—a perfectly average, out of print but not quite rare title that's considered a fundamental work of development biology? Or how about… $23.6 million?  That's the result of a recent bidding war between two third-party Amazon merchants, each attempting to use algorithms to sell an out-of-print version of Peter A. Lawrence's, "The Making of a Fly: The Genetics of Animal Design."  UC Berkeley associate professor Michael Eisen first noticed the pricing irregularities when he attempted to pick up a copy of the title a few weeks ago on Amazon. The book–with a list price of $70 on Amazon–was only available as "New" from two third-party merchants: "profnath," who was listing the title for $1.7 million, and "bordeebook," who one-upped his competition with a price of nearly $2.2 million."

Of course compared to the HFT algorithms, this is a stupidly simple, wildly unsophisticated and incredibly slow example of dueling price algos. The point is that neither individual algo was a problem in and of itself. It was the interaction of dueling algos that created a preposterous price by making dozens of price quote adjustments without human oversight over a period of weeks. Now - Is this at all applicable to the myriad of  proprietary complex pricing algorithms independently competing and seeking to game the market by adjusting prices tens of thousands of times per second at a rate that precludes human oversight?  Fuck Yeah. [H/T to HDW for reminding me of this example]

Which is the Dog? Which is the Tail?

The question is begged: Even if there are feral pricing algorithms running wild in the trading marketplace, operating at extreme high speeds, and partaking in questionable activities - can they really be a danger to the ginormous equity and commodity market?  After all, they are mostly trading in very small increments and  arbitraging fractional price differences across stock exchanges. How big an effect can that really have? Is this not a tempest in a teapot?  Tyler Durdan at ZeroHedge offers this remarkable Nanex animation graphically presenting the impact of HFT activity on every major exchange since Reg NMS made HFT possible.

The Rise of the HFT Machines - by Nanex

This animation shows the daily packet traffic (approximately translates to transactions, quotes, orders, and cancelled orders) at each of the US market exchanges from January 2007 to January 2012.  This covers the period time when HFT became an integral part of Wall Street. You can watch the date scrolling in the lower left corner. The biggest flareup in this animation occurs in August 2011 when US credit is downgraded to AA+. The HFT quants made a lot of money then.  It is certainly fair to ask - Who is really wagging what?

"Kill the Quants Before They Kill Us"

An oft repeated refrain from talking heads on the financial news networks is that we need more human oversight in the pricing process to provide a  common sense circuit breaker for HFT algos.  It was the traders on the floor of the New York Stock Exchange that called attention to the berserk Knight Capital algo and the plug was pulled within 15 minutes of it's assault. In those 15 minutes Knight Capital took $440 Million in trading losses, exceeding their entire cash holdings. In the 36 hours since, Knight Capital has lost 80% and  $800 Million dollars of market capitalization. It seems likely the company will not exist in it's current form by Monday.

The only thing I can think of that is remotely comparable to this notion of human oversight and management of massive number of hyperfast events is the control room of a nuclear reactor.  There human engineers watch data on dials and screens that are hopelessly behind what is actually taking place in the nuclear furnace. The conceit is that engineers can design systems and algorithms that will anticipate every possible thing that can go wrong and these tools will give the human controllers time to react. Most of the time it works. But when it goes bad, it goes really, really bad. The engineers simply did not anticipate everything that could go wrong at Three Mile Island, Chernobyl, or Fukishima.

I am with Doug Kass on this one:

“I say kill the quants and their technology before they kill us,” he exclaims on CNBC’s Fast Money Halftime Report. Kass, who is president of Seabreeze Partners and a CNBC Contributor made the comments with a wink in his eye.  But he’s very serious about events involving Knight Capital and the ripple they will likely send through the markets. Though relatively small – Kass worries they may be the proverbial straw that breaks the camel’s back.“It’s just the latest gaffe in a series nightmares on Wall Street – it’s as if Freddy Krueger is taking and axe to the heart of investors.” Kass points to other technology related woes, such as the Facebook IPO debacle in May, the glitches on BATS in March and of course the Flash Crash on May 6, 2010."

We've had plenty of warning. The lesson of the Knight Capital "glitch" is unambiguous. You can test individual algos all you want but you cannot anticipate every consequence of what may transpire when dueling algos independently fight extended wars for profit within the blink of an eye.

Unfortunately it is not in our collective psyche to react to warnings.  Whether it is the threat of terrorism and 9/11 or the impenetrable synthetic derivatives bundling sub-prime mortgages and the subsequent crash they precipitated - we had plenty of warnings. But we ignored them. We only seem to fix problems after a disaster and a lot of people are hurt.

I'll be linking back to this post shortly after the rogue HFT algo disaster that is in our collective future.

See you then.

Cross posted at Donklephant