This blog post by Xenomorph CEO Brian Sentance is entitled Streamlining EDM Operations: Removing False Positives and was originally published in Traders Magazine. 

Enterprise Data Management teams inherently rely on rules to identify exceptions that need fixing or require further investigation. However, having the right rules in place from the onset is absolutely crucial to ensure streamlined, effective operations.

There can often be a tendency to over-simplify and create rules that are not adaptive to market conditions. For example, flagging stocks whose price has moved more than five percent in a given day may work well in normal market conditions. But what happens when the index crashes by six percent? The rule would flag the majority of stocks as exceptions. Being inundated with thousands of data points to investigate would then undermine the validation process and overload the data management function/team with unnecessary work.

What is needed is adaptive validation that can embed triggers that are relative rather than absolute. Building on the previous example, an adaptive rule would flag stocks that move more than five percent relative to their benchmark. That means a stock that is up by half a percent could be flagged as unusual, but only if its benchmark index crashed by five percent.

Obviously, such an approach need not be limited to one particular asset class. One could build similar rules to flag unusual changes to bond prices – testing how a bond’s yield has moved relative to others of a similar maturity. Or one could test how the bond’s yield has moved relative to the credit default swap spread for the same issuer, or vice versa. Similarly, adaptive rules needn’t just be for price data. In validating volatility surfaces, one could look to benchmark moves in each data point relative to broader measures of volatility – such as the CBOE’s Volatility Index (VIX). Or one could adapt rules to strip out the impact of corporate actions, like the impact on a stock’s price the day it goes ex-dividend.

In all of these examples, the key to building validation rules that facilitate the efficiency and effectiveness of the data management team is that they really do adapt to hone in on true data exceptions, and do not produce false positives when faced with broader market shifts. In order to do that, it is crucial that your data management platform can fully express how static, historic, real-time and derived data are related, and how this data fits within the wider context of the market.