Identifying where a major bank lost millions during an IPO

Market maker unable to publish stock price…

A case study by Velocimetrics

Our client, a leading international retail and investment bank, was acting as a market maker – or broker – for a large IPO that was being issued on the New York Stock Exchange (NYSE).

A market maker is a member firm of an exchange that buys and sells securities at prices it displays in the exchange’s trading system – using these systems a market maker enters and adjusts quotes to buy or sell, executes orders and clears those orders. Disastrously for the bank, its pricing engine froze for a short time on the day of the IPO, meaning an out of date price was published that investors were able to trade against. The bank lost millions of dollars in the space of a few minutes.

Although the client was already using Velocimetrics technology to capture traffic at the time of the incident, they were unable to identify the problem and how it had occurred, as standard test and software tools were not capable of reproducing the issue. As the bank couldn’t recreate the conditions that caused the failure, it wasn’t able to identify the root cause of the problem and stop it from happening again. The bank was under considerable pressure to find a solution, which would help them replay historic market data to prevent history repeating itself.

mdPlay to the rescue…

The onus initially fell with the quality assurance (QA) team for equities trading and there was a real urgency to find a fix. They had identified a problem in production, that had caused the pricing engine failure, and they had to reproduce the scenario.

To do this, the QA team needed a solution, which would help them create a test environment in which to play the market data feed in precisely the same way it arrived on the day of the IPO. Attempts to retransmit market data in the lab using other tools failed, as the problem that occurred on the day of the IPO could not be recreated.

The bank selected mdPlay as it was able to retransmit the captured market data with precise replication of the timing, which is critical for pricing engines and algo trading systems. As a result, the condition that caused the pricing engine to freeze was identified, allowing the team to fix it and put a strategy in place to prevent the same problem from reoccurring.

Using Velocimetrics for the good of the bank…

The bank covers multiple asset classes, including equities trading, fixed income, algo trading and FX trading, but mdPlay is used in particular to capture and replay equities and futures market data.

The solution essentially monitors the production environment and stores every market data message with a precise time stamp. mdPlay takes that data and retransmits it – as it was recorded – into a test environment, with precise replication of timings between market data messages. With this precision, it is possible to reproduce fast moving market conditions and to more accurately simulate behaviour in the real world. The solution continually runs test environments consuming market data – there is one channel that continually plays the market data from the previous working day, allowing for an ‘always on, 24 hours behind’ production testing, for example.

Additionally, this market data capture and replay enables the bank to measure latency and metrics such as tick to trade timings and exchange performance. This gives the bank a holistic view of its trading environment – they know what’s going on in production, they can replicate that in the lab and if they know what the performance is in the lab, the bank can test updates before they are released into the real environment.

mdPlay is also able to do non-arbitrated gap detection and sequenced gap analysis, which enables the bank to discover gaps in different market data feeds and understand how the performance of the different market data feeds varies. This gives the bank the insight to be able to resolve any issues that might be impeding the performance of the data feeds.

Velocimetrics runs mdPlay as a managed service and works closely with the bank’s equity trading technology team and the QA team.

The benefits of using mdPlay

Using mdPlay has had business benefits for the bank, as well as improving its technological capability. Some key benefits include:

Risk reduction: as it enables historic data to be replayed, mdPlay considerably reduces the risk of problems reoccurring as they can be easily identified, as it did in the case of the IPO.
Improvements to QA: the ability to recreate a test environment means the bank can replay market data in the test environment and also enables them to reduce QA costs.
Measuring latency: measuring latency gives the bank insight into the trading process and enables it to put measures into place to reduce latency. They can use the latency measure as a benchmark.
MiFID II test requirements: MiFID II requires financial institutions to test algorithms and mdPlay facilitates this.