Speeding Up Financial Analysis

IBM's combination of hardware and software crunches financial data at an unprecedented rate.

Supercomputers churn through vast amounts of data using multiple processing cores working in parallel. But most supercomputers dip into a database to find information, process the data, and then produce a result--a process that can take minutes or days, depending on the task. In recent years, however, researchers have started to explore the potential of stream computing, a type of computing approach that lets them crunch a real-time stream of data in microseconds. Data from traffic cameras, accident reports, and weather could be used to predict traffic, and streaming audio could be transcribed or translated quicker.

Finding flow: IBM has tested specialized software on its Blue Gene/P supercomputer (above) that analyzes simultaneous streams of data about financial markets. The analysis provides automated trading machines with information 21 times faster than other analysis systems, according to IBM. Credit: IBM

Now IBM has shown that stream computing can be used to analyze market data faster than ever before. The result is a machine that helps automated trading systems determine the price of securities using financial events that have just occurred. To build the system, the computing company partnered with TD Securities, an investment-banking firm, to tweak IBM software called InfoSphere Streams for financial data. The firm ran the software on one of the latest IBM supercomputers, known as Blue Gene/P.

IBM's system improves upon the current type of financial trading systems, which collect data from numerous different sources around the world, including constantly fluctuating prices of stocks and trading volumes. This information is broken into chunks, called messages, which are sent through trading systems. The more messages a system can examine, the more security prices it can determine, the more options can be sold using automated trading machines that match buyers with sellers.

Story continues below

The significant advance, says Nagui Halim, chief scientist of the stream-computing project at IBM, is that the engineers optimized the software to run on Blue Gene/P so that the data streams were analyzed faster than possible on other financial-analysis systems. The information arrived at a rate of five million messages per second, says Halim. The system could process a message within 200 microseconds. The result: a supercomputer that produces security prices 21 times faster than any other financial-trading system.

In some instances, says Halim, it's critical to process the data as it comes in. A system that IBM has built monitors the vital signs of patients, such as their blood gas levels, and keeps track of patient statistics, such as their weight and medication regime. Data from these feeds, which can number in the hundreds, are analyzed and correlated, producing a picture of the patient's health that would be impossible to draw from doctors' or nurses' observations alone.