This blog post by Xenomorph CEO Brian Sentance is entitled Finding the Right Data Model for Enterprise Data Management (EDM) and was originally published in Information Management.

Too Unstructured? Too Rigid? Just Right! 

When it comes to framing enterprise data management, it is essential to establish the right data model. The trick lies in keeping enough structure to capture current business processes while retaining enough flexibility to accommodate ever-changing requirements.

Some might argue that EDM hardly needs a data model at all. All you need is to map data items from input sources to downstream system, adding in validation across sources for each item. Why would you bother considering the wider context of data and what it means to the business? Just keep on adding more data sources, more mappings, more end points. It sounds like a nirvana.

The problem is this kind of approach does not scale well. The complexity of the systems infrastructure (the number of mappings) increases with the number of input sources multiplied by the number of downstream systems. You end up with a tangled mess – a plate full of spaghetti – which is hardly the right material to build your business on.

Also, treating data as autonomous items ignores the fundamental relationships and dependencies between different data sets and business objects. You lose out by not having a centralised body of information (and information about information) for use across all systems. Without that foundation, it is difficult to build out any business logic at all.

By contrast, if you want to capture the relationships and dependencies between different data items, you need a structured data model. Financial data is made up of a myriad of interdependent data points. Having the right data model means understanding those complex relationships. It means knowing how an equity fits into an index, how bond price relates to yield, who has issued what securities or what transactions relate to what positions.

All of the relationships and interdependencies mentioned above can be built into a centralised data model that provides structure for more intelligent, or business-aware, EDM processes. This avoids building up a tangled mess of individual mappings, by developing a common framework of data, what it means and its relationships with other data and business objects.

However, a problem that many systems face – particularly those that use a classical relational data model – is that relationships need to be defined before they are needed. That makes it difficult to add or modify new attributes or instruments, without extensive design changes or enlisting the co-operation of the software vendor involved (which can be slow and costly). So, instead of a tangled mess, you end up stuck with granite blocks – better for building with, but very inflexible, so unless your design is perfect you will suffer the consequences.

So what’s the right approach? The no data model approach, providing no data foundation for the business to grow? Or a data model that provides more structure, but potentially boxes you in with its rigidity?

In my view, the benefits of scale and efficiency from having a data model to build upon are obvious. But it is the rigidity of the classical data model implementation that lets this approach down, not the approach itself. And in the same way, the no data model approach has evolved as a response (perhaps an over-reaction) to this implementation weakness.

There does not need to be any competition or conflict between the goals of structure to build on and flexibility to adapt. Modern technologies now offer data model approaches that build and manage structure, but are designed from the outset for change. Such approaches cope easily with new data sources and new data requirements, but achieve this in such a way as to remove any resulting need for costly database re-engineering or for continual and expensive consultations with your system vendor.