Traditional static data sources are old hat to some of our customers, but many people have business requirements that require visualizing dynamic data. For example, the ability to handle real-time streams is a "must have" in algorithmic trading applications. The developers of these systems often implement support for multiple data layers, including traditional databases and real-time streaming sources. They utilize cutting edge techniques, including messaging middleware, to provide streams of dynamic data to a wide range of different business-critical applications. In my experience, popular third-party sources are most frequently incorporated into algorithmic trading systems that in turn connect directly to low-latency real-time financial applications. We also see developers using homegrown in-house streaming systems with proprietary APIs that are designed to push out updates as they happen in real-time to trading systems, and so on.

CEP vendors are coming into the market with some really interesting systems these days and many financial institutions are beginning to replace their homegrown systems with these vendor-supplied systems. I recommend having a look at http://magmasystems.blogspot.com/ for some interesting posts on the Complex Event Processing topic. Marc Adler, the owner of this Blog, will can give you an interesting take on what is going on in this space. If you follow the Blog you will get a good feel for the mindshare that various CEP vendors are staking out for themselves within the Financial Services market.

Relational and multidimensional databases are familiar and well-known technologies. Their query languages, SQL and MDX respectively, are more or less standardized. Most quantitative and structured data is kept in such static data repositories or will eventually at least end up there. However, this type of quantitative and structured data can only provide part of the whole picture when supporting informed decisions based on data. Ever more unstructured or semi-structured data is required to feed algorithmic trading engines, and this sort of data can be difficult to access even using the latest search engines. And then we have the critical element of live streaming data sources, which is the topic of this post.

How can all of these data sources — structured, unstructured, and streaming — be combined? How can you monitor and analyze them? We have designed our Visual BI platform and our rendering pipeline to support these sources. We use the concept of a data plug-in (also called a connector); this is an adapter that provides access the data, regardless of whether it is static or streaming. These connector modules are available out of the box for some of our products, providing simple point-and-click or drag-and-drop access in applications where our end-users connect to static data sources such as relational databases and multidimensional OLAP databases. We also offer these connectors for a range of dynamic data sources or streams, such as Reuters feeds, Bloomberg terminals and SonicMQ and the ubiquitous Excel format.

What about customers who need to leverage their in-house IP and investments they have made into their own custom built system or a non-standardized proprietary API? By definition, we can’t offer a pre-built data plug-in for these situations, but we can solve the problem. Let’s say the customer doesn’t want pipe the data stream into some standardized data source that we can already access. We can develop a custom data plug-in in cooperation with the customer and then slot that module into one of our existing out-of-the box applications for visual analysis and monitoring. This data plug-in can be embedded into the customer’s existing applications as a component when Panopticon Developer SDK is used to integrate visual analysis and monitoring on top of that app (or to embed it within the app).

We continue to build more data plug-ins to cover more and more dynamic data sources. Please feel free to contact us and let us know what dynamic data sources you would like to see supported in the future!

Markus Skyttner
CTO
Panopticon Software

I introduced Panopticon’s new series of webinars yesterday. Marcus Eder, one of our most senior management consultants, produced yesterday’s event with me. Marcus has over ten years of experience in the implementation of software within financial services organizations and I’ve been working in financial services for over 20 years now, including a stint as a portfolio manager at Putnam Investments.

I’ll start by saying: The integration of static data sources with real-time streaming data is powerful stuff!

The response to the webinar was great! We had over 100 participants from all over the world, including people from the US, the UK, Continental Europe, the Middle East, South Africa, and India. We even had someone from Australia who couldn’t attend since it would have been in the middle of the night for him to watch it live; he’ll watch the recorded version on our website. Based on this fantastic response to the topic, I’m even more confident in saying that interactive visualization tools are the “in” thing for the financial community right now — the most reliable way to gain a true competitive advantage!

In the “What’s My Fill?” webinar, we introduced our tools for Trade Monitoring and showed how they could be used to efficiently visualize the status and performance of their orders on different levels — by sector, trader, country, firm, client, and so on. During the webinar, attendees could type questions to us in a chat room. In the middle of the presentation, one participant from a large financial institution simply typed “So, what IS my fill”? I was a bit surprised at first since that was our main topic for the webinar and we were definitely going to answer that question. What I realized is that he didn’t have time to watch the whole webinar, and why? Because in this market, not only minutes, but seconds separates a winner from a loser. He needed to get back to his business and yet he was so interested in the interactive visualization tools we presented that he took some time to participate in the webinar! That’s why our clients use our interactive visualization tools; they don’t have that (extra) time to spend on reading reports or browsing through tables to find what they are looking for. The information needs to be there, in an instant and in a format that makes it easy to see what you are looking for. This is true for any market these days, whether you’re dealing with commodities, equities, fixed income instruments, CDOs or whatever.

Back in the day, before joining Panopticon, I worked with several large financial institutions and banks. Our main challenges were to find new ways to gain competitive advantages and that was also when I first laid my eyes on visualization tools. That was before streaming real-time was a reality and before there were visualization tools that could be plugged in to any existing system. Today, all I can say is that interactive visualization tools are better than ever before and they will definitely make your life a lot easier. So to answer the question, “What’s My Fill?”, I say that the fill is the tool that gives you that competitive advantage and saves you that extra time. You can use the time you save by using effective visualization tools to give your clients better service. It might sound as a cliché but it’s even more true today – time is money!

Thanks to everyone who participated in yesterday’s webinar and I’d also like to hear from you! Please send me an email (brian.okeefe _at_ panopitcon.com) with your follow up questions. We’re hosting a series of these events over the next couple of months; we’ll announce them on the site and in our newsletters.

Brian O’Keefe
Director of Product Management
Panopticon Software