If visualizations are the “Tip of the Iceberg”, what’s beneath the surface?

Mostly when we first talk to customers, we show them our visualizations and how these can support visual analysis and monitoring based on their business requirements. This isn’t surprising, of course, since the visualizations are very prominent in the GUIs for our software. It may therefore seem surprising that the visualizations actually account for a very small part of our technology – in fact they account for less than a tenth of our total codebase in our product platform.

If the visualizations are the tip of the iceberg, what is beneath the surface? One important and significant part of our technology involves our powerful data model, which is at the core of our offering. It provides rich OLAP functionality that supports fast analysis through the visualizations. The data model is actually a “stream cube” that is capable of doing traditional multidimensional analysis for static data (like those specified by the MDX language) and also for dynamic data streams. It can also be used for monitoring scenarios, triggering rich events when certain conditions are fulfilled.

There are a range of other data models that provide multidimensional analytical OLAP functionality. Why don’t we use one of these, such as the Microsoft UDM? First of all, most other data models are not optimized for real-time performance. Ours is designed for making fast calculations on continuously updating data. Also, our model is designed to match the requirements that live visual interactive displays poses to a data back-end. This involves being able to bind aspects of the data to visuals and other requirements. Most other third party offerings are not specifically designed for such use and start to degrade under heavy streams of live data.

The Panopticon data model conforms to the original rules defined by Codd for OLAP data models by offering support for multidimensional fast analysis. Specific features include:

• Multidimensional conceptual views capable of breaking down dimensions as required by interactive reports.
• Supports intuitive data manipulation, such as interactive direct manipulation (filtering, grouping, and so on.)
• Can be used as a standalone middleware if required, or can be deployed in a client/server architecture – either on the server, on the client or both.
• Can be used both for batch and interpretative use.
• Supports switching analysis models – since the data model is “pluggable”.
• Provides transparency against data sources – provides live access to heterogeneous data sources and can be used to unify structured and unstructured data sources.
• Supports storing aggregates (OLAP results) separately from the source data.
• Supports extraction and treatment of missing values.Provides uniform reporting performance and does not degrade significantly in performance when number of dimensions are increased.

The Panopticon data model can be used on its own to “OLAPify” any data source. Because it provides transparency against data sources, you can also “tunnel” MDX to it; that is, use other “cubes” as data sources.

Since the architecture of our SDK is based on a “pluggable” data model, we can even bypass our data model entirely and tie the visualizations directly to other data models. In this case, the third party data model used must be able to guarantee support for a range of traditional OLAP analysis functions, including grouping, interactive filtering, calculations and some other things expected from OLAP capable data models.

Markus Skyttner
CTO
Panopticon Software