Xenomorph Blog - Data Management

Posts categorized "Data Management"

The Humans Between Risk and Data

Some of my thoughts on risk management, data management and human behaviour, are to be found on page 20 of the Inside Reference Data Special Report "Managing Risk"

Posted by Brian Sentance | 21 June 2010 | 2:22 pm

A Crisis Needs a Utility?

I heard Francis Gross of the ECB speak at one of the panel events at the XTrakter Conference last week, and found that I couldn't avoid asking him whether the aims of the "Data Utility" initiative by the ECB could be better separated from the means by which the ECB proposes to solve them. At the moment, reference data issues for the industry and the data utility seem to be presented as a single "package". I can't say that the response to my question was a clear one to my understanding; however I would say that Francis was helpful after the panel had finished and provided a recent presentation of their ideas, of which you can find a copy here.

Looking through the presentation, the motivations put forward for why the industry needs a data utility seem to include:

Data processing must be done in an automated manner, since data volumes have moved beyond the capabilities of manual processing.
- can't see anyone arguing with this
Data is a major bottleneck, with multiple providers/sources each with the own "data dialect"
- agreed and to some extent what keeps data/data management vendors in business, but sounds sensible to standardise if possible as there are plenty of other problems to address
These data dialects lead to increased cost, operational risk and reduced responsiveness
- agreed, mainly a cost aspect I would suggest
The recent crisis was not helped by weak data management in the industry
- but nor was it the cause, so not a great premise for a data utility
- lack of transparency of data
  - "transparency" is an over-used word at the moment, but certainly clarity and quality were/are needed
- systematic risk could not be assessed due to the availability of data
  - using terms like "systematic risk" seems to imply the regulators could calculate something, whereas this discipline is new so I guess we are really talking about simply knowing who is exposed to who and how.
We need the capability to run large scale computing analysis on a vast pool of micro data, sometimes on an ad-hoc basis when a crisis begins
- fundamentally agreed but also good to qualify with what you propose to be calculated - having a set of "numbers" doesn't seem to have helped much recently...

I started the above bullet point list by saying it contains the motivations for "why the industry needs a data utility" but I guess looking at the above list they really point to the more general aim of "why we need better industry-level data management". In the presentation the above points are then used to state:

"We all need the same good basic reference data. Why build more than one infrastructure?"

Maybe "Why build more than one infrastructure?" should really be changed to say "Why maintain more than one infrastructure?" given that Bloomberg, Thomson Reuters, Six Telekurs, Interactive, Markit and all the other vendors already infrastructure to do this. Not sure if I should read anything into the wording but more logical leaps of faith are to follow.

The presentation then moves on to state that shared reference data standards are a must, to which I cannot see many consumers of data disagreeing with that statement. Not sure I agree though with the overly simplistic statement that "Data will be good for all users or good for none". Trying telling that to the accountancy and risk departments for example but I suppose what we are talking about here is basic reference data not the more subjective price and valuation data. Reference data on instruments and entities is either right or wrong, and the presentation makes the good point that no amount of "data cleaning" can help this i.e. if wrong, the data needs to be re-captured from an accurate source.

The call for the establishment and use of reference data standards in the presentation then seems to be used to "slide "into a call for a standard reference data infrastructure. Unless I am very much mistaken, these two things are not necessarily the same thing and so it seems a logical leap has been taken here. The presentation talks about the possible necessity of "top down" legal compulsion for the industry, again something that I could agree and see the need for, but both the issues and legal compulsion do not automatically drive us to a "data utility" as the only option? Why couldn't legal compulsion be applied to the existing data vendors to standardise on common IDs for instance? ISIN is proposed as a standard in the presentation, but I can only assume that this is due to the ECB being mainly focussed on the bond world where to a large degree ISIN's work (i.e. are unique), whereas in the world of equities ISIN needs a lot of qualification (currency, exchange, share class...) before it uniquely identifies a quoted equity.

In summary, the presentation starts with showing how great the ECB's Centralised Security DataBase is (7 million securities, 3 million record updates/day etc...) and it does look good. The data issues for the industry seem clear, although I think the "crisis" is a bit of a red herring to the aim of data cost reduction, however the logical jump from industry need to effectively "we must have a data utility" is an interesting one, one where I would prefer that more options were discussed. It seems ironic that in these days of "transparency" it is not at all that transparent to me why more alternative solutions are not being discussed and a choice justified. Talking of choice and as a final thought, I am also not sure why the data vendors are not up in arms about this initiative - are they frantically lobbying behind the scenes? - do they simply think the utility won't go ahead? - or are they afraid of upsetting the EU? Any insight is very welcome, and maybe more of update from me when I get chance to speak with Francis in more detail.

Posted by Brian Sentance | 4 June 2010 | 8:00 am

XTrakter Conference

I went along to the XTrakter Annual User Conference in London on Thursday - Good event with some great speakers. Angela Knight, CEO of the British Bankers Association, gave a talk to start off the day. Angela seemed a lot less on the defensive than when I have heard her on national radio here in the UK, usually being interrogated by some journalist who wants answers to difficult questions on the financial crisis and the banks role within it.

Angela said that we were in year 3 of the "crisis" with 2008 being about the banks, 2009 being about governments and politics and 2010 being the year of sovereign debt. I guess she enjoyed saying this but that everyone is blaming "Anglo Saxon Banking" for our problems and yet it was not the banks that contributed to the fundamental problems that Greece is facing.

One major theme of her talk was decidedly Euro-Sceptic in tone, which was that the UK idea of internationality and international trade was different from that of Europe. She perceived that in the UK one of our trading parties is Europe, whereas international trade in Europe was more about inter-European and not world-wide trade - I think that there are elements of truth in this but not sure that Germany industry for example would agree that it is not conscious of truly "global" trade? She said that she was concerned by the rules and regulation being put up by governments, particularly in respect of there being too much and in too short a time.

Angela was an engaging speaker and at the very least her opinions prompt reaction, however I have to end this quick post with the best quote of the morning from Anthony Belchambers, CEO of the Futures and Options Association. Anthony said that current frenzy around political and regulatory initiatives to control the financial markets remind him of:

"A bar room brawl, where the brawlers don't punch the person that started the fight, they punch the person they have always wanted to punch..."

Posted by Brian Sentance | 24 May 2010 | 4:11 pm

Counterparty Event

I went along to a morning panel on counterparty data management on Tuesday, sponsored by GoldenSource, Avox and Interactive Data, and hosted by Virginie O'Shea of the A-Team. Counterparty data obviously has a very high profile currently in light of recent events, however the advice from the panel fundamentally seemed to be get the basics of data management right (ownership, control, consistency, quality, transparency), rather than anything radically new.

There was some debate about the possible extension of BIC (Bank Identifier Code) to be used more generally as a standard for a unique business entity identifier - this seemed to be received well but there were concerns that such an initiative would not solve the problem but rather become an addition to the already complex entity-mapping process.

The "Data Utility" from the ECB was also debated, and it was refreshing to here some negative (realistic?) things said about it, such as the concern raised by Interactive that this might involve huge public spend without necessarily understanding why a new government sponsored entity would be able to do better than existing data providers. Obviously a data provider would say that, but I have to agree, it seems there is too much focus on having a data utility and not looking at the different options for solving industry data issues (one option obviously being a data utility, but lets not pre-package the problem with a solution but more of that in later posts...).

For more detail on the event, then take a look at Virginie's blog post.

Posted by Brian Sentance | 21 May 2010 | 10:56 am

Cloudy definitions

Given that I am English and can tend to start many personal introductions with a short conversation about the weather (generally either "awful" or "not bad for this time of year"...), then maybe I should be very receptive to the use of weather-related expressions in technology such as the "cloud". Maybe not however since the "cloud" and "cloud computing" have reached that zenith of marketing hype, when everyone is talking about a new technology regardless of if they are sure what it actually is (or might be, or could become...).

Anyway, I finally swallowed my cynicism and on Thursday morning went along to "Migrating Business to the Cloud", an event by Microsoft hosted at Bafta (small venue where the UK deals out its equivalent (?) of the Oscars). The master of ceremonies was Mark Taylor of Microsoft, who gave a general introduction to what Microsoft are doing in the "cloud", and of particular note he described the four types of computing scenarios where cloud computing can optimally be applied:

Predictable Bursting - where computing needs come and go in predictable waves of usage/demand
Growing Fast - where computing needs are rising exponentially like in a successful internet start-up
Unpredictable Bursting - where computing demand comes in unpredictable bursts, such as that associated with say usage of a backup computer centre in disaster recovery
On and Off - where you might run a process once a month or at an interval you decide

The above definitions seem ok to me but there is (probably understandably) some overlap in usage cases. The "Growing Fast" case for start-ups is interesting and more of that later.

Mark handed over to David Chappell who gave his perspective on cloud platforms as they are today in the market. David was a very entertaining and knowledgeable speaker, despite wearing a dodgy suit (what happened to those trousers?!) and having a peculiar wide foot stance when speaking. Anyway I digress, on to what he said. David started by saying what the "Cloud" is comprised of:

Cloud Applications - basically this is Software as a Service (SaaS) and some current examples of this would be Salesforce.com CRM, Microsoft Exchange Online and Google Apps.
Cloud Platforms - a platform for developing cloud applications, with the following characteristics that it:
- is aimed at developers for creating and running cloud applications, not end consumers
- provides self-service access to computing resources
- allows very granular, on-demand allocation of computing resources
- charges for the consumption of computing resources in a very granular manner

David then explained that due to its ambiguity he disliked the usage of the term "Private Cloud" in the ongoing debate about publicly available cloud services (such as those provided my Amazon, Microsoft and Google) vs. private clouds deployed within private institutions. David said the main difference was that private clouds do not have the economics of public clouds (i.e. pay for what you use only when you need it). That point seemed straightforward, however I would have thought that with a large global organisation with many different departmental computing demands the economics of a private cloud would be similar to a public one.

David then went on to explain that there are two kinds of Cloud Platform:

Infrastructure as a Service (IaaS) - this is a cloud platform the provides a developer with a virtual machine (VM) that has (almost) full access within it; put another way the development environment gives the developer total control but with that control comes responsibility.
Platform as a Service (PaaS) - this is a cloud platform that runs an application that a developer has created; it is easy to use but has limited control for the developer.

David put forward that there has been only 5 major software technology platforms over the past 50 years:

Mainframe
Mini-Computer
PC
PC-based Server
Mobile

He perceives that the Cloud is the 6th major software technology platform, and as such he is extremely enthusiastic about the opportunity and benefits that this presents to the whole of the software industry and its consumers.

David categorised Microsoft's cloud platform as (mostly) PaaS, which had three main components:

Windows Azure - for environment for running cloud applications within the platform
SQL Azure - relational storage within the platform
Windows Azure Platform AppFabric – (David noted the long name and sympathised with trying to name things sensibly) this provides and manages the infrastructure within the platform

He then moved on to describe the main usage scenarios for Windows Azure, for applications that:

need massive scale, such as Web 2.0 applications
need high reliability
have highly variable loading
have short or unpredictable lifetimes
need parallell processing
will either fail fast or scale fast
do not fit easily in a single organisation's data centre, such as joint venture
need external storage

David said that in the fail quickly or scale quickly scenario, this was squarely aimed at technology start-ups where using Cloud technologies would effectively increase the frequency at which new ideas could be tried out at less economic cost if they go wrong, but are ready to scale massively if they become the new "Facebook" - so much so that many of the VCs in Silicon Valley are now insisting that start-ups use cloud technology as a condition of funding.

Amazon's Elastic Compute Cloud (Amazon EC2) was the first major commercial cloud platform, and David categorised this as IaaS, where effectively you get a Virtual Machine (VM) environment that provides a lot of control but requires more effort to control than an PaaS such as Azure.

David said that he was surprised that the Google App Engine, which has Python and now Java as its programming languages, did not come with any traditional relational storage (unlike most other cloud platforms) but on speaking with Google he found that the storage engine and the whole platform is again designed primarily for Web 2.0 apps and as such storage usage was more about retrieving photos, video etc and less about querying across many records.

David was very complimentary about the cloud platform from Salesforce.com called Force.com, He said that the sales pitch from Salesforce.com would be straight to business users, effectively saying that they could build scaleable, resilient applications without involving the IT department and without needing programming expertise. He asked the audience if anyone had used these tools and a few folks confirmed that they were extremely impressed by what the platform offered.

Bob Muglia (President, Server and Business Tools, Microsoft) then gave a quick talk on Microsoft's plans for Azure. He mentioned how Microsoft's new search engine, Bing, was based on several hundred thousand servers running in Azure, but only had a handful of operating staff in contrast with the usual economics (taken from Gartner) that usually 1 operations person was needed for every 50 servers. He emphasised that Microsoft was committed to the further development of "on premises" operating systems but that Microsoft was totally committed to cloud computing, its development and its support.

He said that some of the tools found in the Microsoft technology suite, such as SQL Reporting Services, are not yet available in the cloud on Azure/SQL Azure (due end of year though) - he said that he hoped that people understood that re-engineering an existing application for the cloud sometimes took time to ensure the scaleable and reliability demanded when providing the functionality through the cloud. The vision put forward by Bob for development of cloud applications seemed very compelling, with Microsoft aiming to make things such enabling resilience for a globally available cloud application as simple as ticking a check-box in Microsoft Visual Studio. He put forward that the major barrier to cloud adoption was the human aspect of trust of moving applications "off premises". He said that he saw a fundamental shift across all industries to cloud development and deployment, but added there may be some areas such as government and finance where this process takes a lot longer.

The event then switched to presentations by EasyJet, RiskMetrics and SeeTheDifference. The head of IT at EasyJet gave his pitch first. His department get an annual budget of 0.75% (small?) of turnover of £2.5bn (larger, so translating to £18.75m) and has around 60 people. He presented how EasyJet has taken an incremental approach to the adoption of cloud computing, utilising both "on-premises" and cloud ("off-premises") technology together (exposing end points of applications into the cloud at first). He advised this approach since it:

was a smaller step than full-blown adoption
was lower risk
demonstrated big value in a short time-frame
leveraged the rich functionality available in Azure
accelerated acceptance of cloud technology

Dr Rob Fraser of RiskMetrics was next up. He explained whilst Moore's Law says that computing power doubles every 18 months, the calculations needed for risk management have doubled every six months. This has driven the need for parallel computing to meet this calculation need, and that RiskMetrics' RiskBurst service uses around 2,500 64-bit Opteron cores in their data centre but combines this with use of Azure to meet the peaks in calculation needed during each day (the similarities with power consumption management were pretty apparent). He said that average CPU consumption was around 18% of peak, hence a combination of both on and off premises compute power was a good solution for them. He mentioned that the management of this hybrid combination of technologies, and in particular being able to show real-time billing for it was a key area of investment for RiskMetrics.

The final presentation was by SeeTheDifference. The main point of this presentation was that this charitable organisation had zero permanent staff involved in IT, but regardless was able to deliver a very professional, reliable and scaleable website using external consultants to build on Azure.

Final section of the morning was a roundtable discussion with questions from the audience. The EasyJet guy said that the human mindset was key to the adoption of cloud computing. In terms of what keeps him awake at night was the thought that what would happen/how would attitudes change if any of the cloud infrastructure failed - so far it has experienced 100% up time. Rob of RiskMetrics was concerned about the stability of the platform, trying to ensuring that any changes introduced do not damage reliability. He added that he disagreed with Bob Muglia and thought that financial institutions would adopt public clouds quickly – he cited their experience of their revenues now being 90% based from service provision not on-premises applications. David said that he took some of the comments from Bob to indicate that Microsoft would also offer more of a pure VM (IaaS) soon in addition to the PaaS approach of Azure. David said that trust was the major issue in cloud adoption and he advised an incremental approach so "get your feet wet" then build from there.

On the whole the presentations were good and my knowledge of cloud technology has improved a bit - certainly it is fantastically appealing to develop globally available applications with no scaling, no resilience or data replication issues - it sounds too good to be true which generally means it is, so I guess there is much more work to do in gaining trust and acceptance for this technology. So my (pragmatic?) cynicism remains - but cloudy days are certainly coming and for a change maybe this is something to very much look forward to.

Posted by Brian Sentance | 17 May 2010 | 9:37 am

CEP - Part of the technology furniture?

The CEP market is apparently maturing - don't miss this post "CEP: LaserDisc or DVD?" by Adam Honoré at Aite Group with an interesting view of the future of CEP technology.

Posted by Sara Verri | 29 March 2010 | 12:29 pm

Data models are not what they used to be...

AIM have released the results from their 2009 survey on reference data management which is worth a look, particularly given the 2008 results are also shown for comparison. Seems like Mike Atkin and the EDM Council have their work cut out in getting the Semantics Repository adopted if the survey is anything to go by, with the number of institutions using standards-based data models having dropped significantly when comparing 2009 to 2008. What is going on there in these heady days of the finance industry sorting out its data problem through adopting standards? - In cash starved times, maybe it costs more to conform to a standard? - Is the survey data not broad enough? Any ideas appreciated!

Posted by Brian Sentance | 18 March 2010 | 8:09 pm

Risk, Data Transparency and the MBS Market

I spent the morning yesterday over at the FIMA USA event in New York, and caught the panel discussion chaired by Neil Edelstein of GoldenSource. Stand out speakers were Amy Hawkins of BNY Mellon and John Bottega of the Federal Reserve.

Neil started the panel by asking the panel for their thoughts on the current drive to improve "data management for risk". Transparency and quality were mentioned a lot unsurprisingly, with John Bottega adding that he was aware that a lot of banks were now focussed on the data that in the past had been "not available" for risk management, not just the quality of data that is readily accessible. All panelists focussed on the need to manage risk across the whole institution, not just by product silo.

On the topic of data standards and transparency, John referred the audience to testimony on the Mortgage Backed Securities (MBS) market presented to the US Government by the XBRL group. Apparently the filing process for mortgages allows free format filing and so is of little use from an automated processing point of view. John also pointed out that a key piece of data in assessing risk is that the "first time buyer" flag was found to be present in only 15% of the filings.

John also mentioned that if loans and mortgages could be given standard identifiers, then this would enable new levels of risk management - for instance it should be able to extract those obligations against a specific region that for example is experiencing economic recession. These would be the benefits of getting data standards in place.

As was later expanded upon in a later talk by Kay Vicino of Northern Trust, there was a lot of panel discussion on organisational data governance and the management structures needed to achieve it. On the governance side of things then whilst it is not an exciting topic, it is obviously vital - main point seems to be establishing data ownership and responsibilities which brings me back to the point that a lot of (most?) data management issues are down to managing people and organisational politics, not just down to good technology (although it helps!).

Overall a reasonable panel, and the XBRL testimony looks worth a more detailed read (if the testimony link doesn't work then go to the www.xbrl.org site and search for a report called "Using Standards for Transparency")

Posted by Brian Sentance | 17 March 2010 | 4:42 pm

Data Management Panel

Thomson Reuters held a panel event on data management at their London offices on Tuesday last week, with speakers from Barcap, LCH.Clearnet, DB, Mizuho and Citi. This event was held in follow up to their recent report "Beyond Golden Copy". Below are some of my notes on the summary points the panelists made:

The Value of Data - Kris Bhattacharjee of Barcap said that there were currently two main drivers behind the perceived business value of data; i) Regulators are expecting more information, adding additional requirements and conducting more adhoc reporting requests. ii) Business users/decision makers want more granular understanding of trading and risk management data, in order to decide how best to allocate scarce capital to what trading positions.
Data Metrics - Kris said that the metrics were many but timeliness of data was becoming a key metric - over the past two years regulators have moved from allowing say 2 months as a reporting timeline down to 10 days recently. Additionally timeliness is again vital as regulators demand adhoc reporting in response to market events.
Accuracy/Completeness - Again regulators are driving this, with the "bad numbers in, bad numbers out" as the main motivation. Unsurprisingly, counterparty data is also being required at a new level of detail and accuracy down to a portfolio level in light of the crisis.
Granularity of Data - Deeper granularity of data being driven by scarce capital and the need to understand how efficiently it is being used. Basel II has also driven greater granularity over Basel I. Reflecting what I have heard from some our clients, Kris added that the data associated with securitised products had increased greatly as people need to understand exposure/risk and pricing in more detail (rather than assume blanket statistical behaviour for a whole basket of assets).
Stress Scenarios - Kris again mentioned the understanding of counterparty exposure driving the need for new data sets, as had the initiative of banks having "living wills" to allow a bank to be wound down in an orderly manner.
Everybody has Left the Building! - Martin Taylor of LCH.Clearnet was a great speaker and said that the biggest new problem that the collapse of Lehman's created was that ordinarily there are people around to help with extracting from systems what the exposure is to the various counterparties. In the Lehman's case there was nobody around to help, making the process very difficult and leading to the need for changes to address this problem.
Mandating Data Integrity - Martin added that data security, integrity and auditabiliy were vital, and in particular put emphasis on the people that are running the systems that they have their own form of integrity so that an institution knows that the people can trusted but is also capable to deal with a situation where the people are not around to help. Martin felt that this level of data management should be mandated on the industry and that there was an awful lot that finance could learn from industries such as Pharmaceuticals in terms of product approval and management/robustness of data.
Data with No Cost or Value - Neil Fletcher of DB was another good speaker who started his talk by saying that pre-crisis people thought of data as project based, otherwise dealt with it on an adhoc basis and considered data as having no cost or value. Institutions had a spaghetti approach to data, with systems/projects being process not data based i.e. the systems get only the isolated data sets they need only when they need it.
Quality is Now the Data Driver - Neil said that 18 months on from the crisis, then whilst ROI is still important for data projects then quality of data is the key driver.
Sponsorship and Ownership of Data - Neil added that quality data is an asset as are the systems that produce data quality, and to ensure success data management projects needed high level business sponsorship, but also ongoing and clearly defined ownership of all data sets and their quality.
Enterprise Data Virtualisation - Neil said that DB were embarking on a long term project to ensure that all systems get data from the same logical place on a global basis, and that they were investing heavily in data virtualisation technology as a key means of achieving this goal. DB are starting with reference data, moving to transactional/positional data and on to other data types. For each type/category of data ownership would be clearly defined across all systems and would enable real-time transformation of the data into whatever format it is needed in.
Enterprise Data Model - Neil said that as a result of this virtualisation approach then you have to invest in putting together an enterprise data model for all data used in an institution. From my point of view this could be interpreted as a move back to "big EDM" (with all the project risk that implies) but I guess it is being approach on a more staged manner.
Lip Service to Data has Ended - Neil summarised by saying that lip service to data management has ended with the start of the crisis and that 18 months on the enthusiasm for dealing with the data problem has not diminished.
Publish/Validate/Subscribe - Simon Tweddle of Mizuho echoed a lot of what Neil said in approach to global data management and ownership, but added that he believed that the model of publish/subscribe needs to change to publish/validate/subscribe to ensure data quality.

Most of the panelists agreed that bringing in experience from external industries (Pharma, Oil & Gas, Internet Search etc) would be beneficial since we should not assume that the financial market has the expertise to get data management right first time (take a look at this article from the FT for a related idea). Martin of LCH.Clearnet was convinced that mandated data management would come and would be beneficial, which some of other panelists did not agree with and suggested that the industry needs to get ahead of the regulators to head this possibility off. Simon said that the focus on complex data/products was wrong given that the basics (what is our exposure to this counterparty?) were not being done (not sure I agree with this totally, both are needed given the losses from CDOs etc). Overall it was good panel with some interesting debate and speakers.

Posted by Brian Sentance | 8 March 2010 | 2:30 pm

Beyond Golden Copy?

Interesting reading in a survey put together by Lepus and Thomson Reuters and publicised on Finextra this week. Summary findings:

Data management budgets are increasing, with 77% of firms intending to increase spend on data quality and consistency and 32% saying spend would increase significantly.
Tearing down data silos is a key initiative, 70% of firms are looking to revise data management solutions as a result of the crisis, and 31% of firms cited data quality and consistency as the most important driver.
Data management for risk is the top concern, with 87.25% of firms looking to integrate data repositories in risk, and 62.5% saying that they were close/very close.

This seems to be consistent with another article on Finextra this week, with Deloitte predicting a much greater spend on risk management projects. Putting the marketing aspects aside for a moment, I don't think it is abundantly clear from the actual content of the Lepus survey as to why the title includes the phrase "...Beyond Golden Copy" other than the type of data management they refer to seems to have more emphasis on global/firm-wide data integration than your traditional EDM golden copy data warehouse approach.

It is also interesting to hear so much about consistent data across the entire enterprise (driven by risk and regulation) which seems to echo the "big EDM" projects of old that did prove that successful, and to some degree is at odds with what the likes of Golden Source and Asset Control are currently saying about choosing smaller projects to bite off on rather than the enterprise approach. I would suggest however that there is no issue in having smaller projects in mind so long as they are compatible with the overall goal.

The integration and consistentency of data across front, middle and back office was also interesting, and in particular the front office integration echos some of the things I have been saying about the need for analytics management and the management of front office data as part of the data management process, not something to be ignored in the hope it sorts itself out.

Posted by Brian Sentance | 5 March 2010 | 3:28 pm

Fund administrator or data distributor?

Just caught up with this article appeared on the A-Team website - Bloomberg is facing pressure from the industry with regards to users concerns about its initiative to make its codes freely available (see previous post Truly "Open" Bloomberg?). In the article, Max Woolfenden, managing director of FOW Tradedata, recognizes the potential of the BSYM website but advocates more progresses to be made in order to improve completeness of the data offered and in particular to clarify what exactly 'open' means.

According to A-Team, Bloomberg is also facing pressure with regards to a possible introduction of a new licensing structure for Service Provider Agreement (SPA) contracts for fund administration clients. Under the new system, fund administrators would be required 'to pay per security in each individual client portfolio', effectively changing the status of the fund manager to that of data re-distributor with all the cost increases that implies. It will be interesting to see where this heads - will the administrators simply pass the data costs through to their clients, absorb some costs as a competitive play or simply move away from using Bloomberg data?

Posted by Sara Verri | 23 February 2010 | 5:28 pm

More CEP Events

Sybase have acquired Aleri according to Finextra. It was less than a year ago when the complex event processing (“CEP”) vendors Aleri and Coral8 announced their merger (see press release); there was also a big buzz when Sybase announced a CEP capability based on Coral8 and Streambase decided to offer an Amnesty Program for Aleri-Coral8 Customers (see earlier post 'Merging in public is difficult...). And only a few months later, Microsoft announced that their CEP Orinoco (now integrated with SQL Server 2008 as StreamInsight) was heading to market (see post 'Microsoft CEP surfaces as 'Orinoco').

Another sign that CEP is moving more mainstream and that real-time everything is becoming more important? Or a good market for acquisitions?

Posted by Sara Verri | 4 February 2010 | 6:00 pm

"Cut and Paste" Valuation Services

You can talk about more robust modelling, more stringent scenario testing and even moving everything onto an exchange, but unless we move the principles of good data management (in my view: consistency, security and quality of all types of data) into the front office then we will continue to get front-office mis-marking as described in this article in the FT.

Thanks to Ralph Baxter from Cluster7 for highlighting this article for me and those of you interested in this topic of operational risk and spreadsheet mis-use should maybe go along to EuSpRiG this year, and maybe take a look at a paper Xenomorph presented at a previous conference.

Posted by Brian Sentance | 4 February 2010 | 9:49 am

Views on Fair Value...

Busy week last week for events in London, this time over at the Goodacre / Six Telekurs on Thursday morning. Guy Sears of the IMA was chair of the event, and the event did have a "buy-side" focus to it. Richard Newbury of Six Telekurs started the event and made the following points on the current state of regulation:

UCITS IV - Richard cited the stats that there are around 37,500 funds in the EU with average value of approximately $180M each as compared to only 8,000 funds in the US with average value over $1B. Richard said that such a proliferation of funds was costly and the more EU could standardise funds and their ability to be transacted everywhere in the EU the better.
Reg NMS - Richard took a little humorous dig at US regulators when he reminded us that Congress authorised the SEC to form a "National Markets System" in 1975 and so this had taken around 30 years to implement. Whilst Reg NMS is often compared to MiFID, he said that Reg NMS had led to consolidation in the US while obviously MiFID has led to fragmentation in the EU.
Hedge Funds - Both EU and US regulators are looking at the hedge fund industry. He mentioned the battle the UK was having with some of the (misguided?) regulation that the EU is trying to introduce with over 30,000 HF related jobs in London. The new regulation is likely to increase reporting requirements leading to more need for regular, standardised fair value reporting.
Credit Rating Agencies - Richard mentioned how there will be more ratings and more ratings types, and the regulation introduced to ensure the CRA do not fall into the conflict of interest trap.
Data Management - He mentioned the importance of data management within what is happening in the industry and noted how the profile of data management was on the increase.

Mike Jenkins of Ernst & Young tried his best to make the accountancy treatment of derivatives interesting and didn't do too bad an effort but I only took the following few notes from his talk:

Unlike US GAAP with FAS 157 there is no single standard Fair Value (FV) definition in IFRS, and unsurprisingly IASB are addressing this.
Mike spent some time mentioning Level 1(quoted), Level 2 (observable) and Level 3 (unobservable) pricing inputs for securites, taken from the IASB exposure draft ED/2009/5 (also see Rowe in earlier post)

Matthew Cox of BoNY Mellon Security Services then gave his presentation on the difficulties/challenges of providing a valuation service to their asset management clients:

His division often have a "2 hour" window to produce valuations for NAV reporting, often for a 12 midday valuation
Data exceptions for investigation went through the roof this year due to increased volatility (comment: didn't get chance to ask whether the validations set were "normalised" for market volatility i.e. a price movement threshold would not be fixed but rather be multiplied by a factor relating to recent volatility levels)
Matthew was very complimentary about the efforts his team put in to cope with this increase in data exceptions.
He mentioned how many of his clients of established "Fair Value Committees" over the past couple of years, comprised of staff from compliance, risk management, portfolio management etc.
Matthew mentioned the importance of time zones in valuation and the timeliness of data, with the availability of intraday CDS prices contrasting with bonds who price only from the evening close of the day before.

The panel debate was moderated by Guy Sears, and included the above speakers plus Nigel Reynolds from TD Waterhouse):

Matthew said that his division sometimes shared the "consensus" price from other clients when one client is looking for some guidance.
He mentioned that a key timeframe in establishing FV was establishing what is a "reasonable" time frame for sale of a security.
Nigel Cox said that "suspended stocks" had been a real issue over the past year, where the client "context" (position, situation etc) would very much determine what value a client would want assigned to a holding.
Guy Sears suggested that valuations should be provided with a confidence interval and not just as a single price
Mike of E&Y said that this is what full disclosure now requires, other memberrs of the panel suggested this was realistic but not what clients (humans?) expect to receive - they want a single number.
Guy wondered whether it was an issue that one entity might value an asset at a value X whilst another would value the liability at Y (not equal to X)
Mike of E&Y pointed out that this was an issue in that current accountancy rules allow a security to be reclassified from "fair value" pricing to "historic cost" basis - this discretion is being removed in future rule implementations
One member of the audience pointed out that Bloomberg, Reuters and Markit were all trying to extract more revenue from data used for valuation purposes.
Matthew advocated that the market needed more competition between niche data vendors such as Markit and SuperDerivatives to ensure innovation in service and more competitive pricing.
The audience asked Guy of the IMA whether the association should have offered more guidance on fair valuation process and best practice.
Guy said they have provided some, but he advocated that trade associations should not have opinions, since it was not healthy to have the asset management industry collectively herding towards the same valuations.

Well attended event with some good speakers, particularly Guy Sears as host was funny, knowledgeable and kept the other speakers on their toes. I would say the most interesting point was still that "opinions" form prices, opinions formed in the investment/funding "context" of the party with an interest in valuing a security - conceptually this seem to make the asset servicing companies a little uncomfortable since what they are contracted to do is to provide the "right" set of numbers by their clients. Human beings feel more comfortable fixating on a single number than a range of possible outcomes/results it would seem!...

Posted by Brian Sentance | 17 November 2009 | 10:48 am

It's in the news...

I went along to the Forum on News Analytics over in Canary Wharf on Monday evening, organised by Professor Gautam Mitra from OptiRisk / Carisma at Brunel University. We seem to be in the early days of transforming news articles into quantifiable/machine-readable data so that it can be processed automatically/systematically in trading and risk management. It was a good event with both vendors and practititioners attending so was reasonably balanced between vendor hype and the current state of market practice.

As background on what is meant by news analytics data, then for example you might count the number of news articles about a particular company and look at whether the quantity of news articles might be a predictor of some change in the company's stock price or volatility. Moving on from this simple approach (assuming that you are clever enough to be certain about what news is about what company), then you can then move towards assessing whether the news is negative, neutral or positive in sentiment about a company/stock.

The context here is about having the capability to automatically process/analyse any kind of text-based news story, not just those from research analysts that might be nicely tagged with such quantifiers of sentiment (see http://www.rixml.org/ on xml standards for analyst data). The way in which the meaning of the text is "quantified" uses some form of Natural Language Processing.

The event started with a brief talk by Dan di Bartolemeo of Northfield Information Services. I hadn't heard of him or his company before (maybe I should pay more attention!) but he seemed a very solid speaker with strong academic and practical background in investment management and modelling. He referenced a few academic papers (available via their web site) on news analytics, and how news analytics and implied volatility could provide better estimates of future volatility than implied volatility alone. He also made some good points about how investment "models" are calibrated to history and how such models need to adapt to "today" - he put it as "how are things different now from the past?" and put forward the idea of a framework for assessing and potentially modifying a model to respond to the "now" situation. He also suggested that the market can react very differently to "expected news" (having a range of investment "what ifs" planned for a known earnings announcement) as opposed to unexpected information (we are back into the realms of the Black Swan and the ultimate in uncertainty wisdom from Donald Runsfeld)

Armando Gonzalez of RavenPack then began by explaining how RavenPack had become involved in applying text analysis to finance (it seems the subject has its origins, like a lot of things, in the military). RavenPack seem to be highest profile quantified news vendor at the moment, and whilst Armando is obviously biassed towards pushing the concept that money can be made by adding quantified news data to trading models, he said that not many firms are as yet systematically processing news and most people are relying upon manual interpretation of the news they buy/use. Some of the studies Ravenpack have on market news and prices are very interesting, showing how a news event can take up to 20 mins before the market settles on a new "fair" price level for a stock. Additionally, and maybe an interesting reflection on human behaviour, was that in bull markets there are usually twice as many positive stories about companies than negative, but strikingly in a bear market there was still almost equal amounts of positive and negative news - so humans are basically optimists! (or delusional, or just plain greedy...take your pick!)

Mark Vreijling of Semlab followed Armando and suggested that a lot of their sales prospects understandably desire "proof" of the benefits of adding quantified news to trading, but this was a little ironic since most financial institutions have been paying to receive "raw" news for years, presumably because they perceive beneift from it. Mark also mentioned that the application of quantified news to risk management was a new but growing area for him and his colleagues.

Gurvinder Brar of Macquarie then went into some of the practicallities of quantifying and using news in automated trading. He suggested that you need to understand what is really "news" (containing information on something that has just happened) and what is merely an news "article" (like a "feature" in a magazine etc). Assessing relevance of news was also difficult and he added that setting a hierarchy of what kind of events are important to your trading was a key step in dealing with news data. Fundamentally he suggested that why wait for five days for analysts to publish their assessment of a market or company-specific event when you could react to the event in near real-time.

The event then went into "panel" mode where the following points came out:

Dan thought that a real challenge was integrating quantified news with all of the other relevant datasets (market data, but also reference data etc)
Armando picked up on Dan's point by giving the example news about Gillette which at one point was about Gillette the company but then on acquisition became news about the Gillette "brand" which became a part of Proctor and Gamble.
Dan said that a key problem with processing news was also understanding what news was simply ignored by the news wires i.e. we know what is being talked about, but what could have been talked about, why was it ignored and is it (even so) relevant to trading?
Mark and Armando said that the "context" for the news story was vital and that market expectations can turn many "negative" news stories into positive outcomes for trading e.g. the market likes bad news when it is not as "bad" as everyone thought.
Dan made a very interesting point about trading in terms of categorising trades as "want to" trades and "have to" trades. He gave the example of a trade being observed that seemingly has no news associated/prompting it - so does this mean the trade is occuring because somebody "has to" make the trade (a fund facing an welcome client redemption for example?) or because there has been some information leak to a market participant and such a participant "wants to" make a trade before the news becomes available to the market as a whole.
I think all of the panel members then collectively hesitated before answering the next question from the audience, with Microsoft having one of their "text search" R&D team (think Bing...) asking about news categorisation and quantification.
Dan also mentioned something that I have only recently become more aware of, which is that apart from major markets in the US, most exchanges world-wide do not publish whether a trade was a "buy" or "sell" trade (they just publish the price and transaction size). Obviously knowing the direction of the trade would be useful to any trading model, and Dan referred to this as wanting to know the "signed volume".
A member of the audience then asked whether most quantified news had been based on just the English language and the concensus was that most was based on English, but Natural Language Processing can be trained in other languages relatively easily. A few members of the panel pointed out that all languages change, even English, requiring constant retraining, and also that certain languages, countries and cultures added further complication to the recognition process.
The next question asked was whether the panel could outline the major areas that quantified news is applied in - the answer included intraday (but not quite real-time) trading, algorithmic execution, lower frequency portofolio rebalancing and in compliance/risk/market abuse detection.
A good debate ensued about whether "news" was provided by the official newswires or by the web itself. The panel (and audience) concensus seemed to favour the premise the news wires are the source of news and the web is a reflection/regurgitation of this news. That said, Gurvinder of Macquarie gave the nice counter example of the analysts/news wires not making much of the new Apple iPod, when looking at the web it was possible to see that the public were in contrast very enthusiastic about it.

Overall an interesting event. I think the application of "quantified news" to risk management is interesting - maths and financial theory is very interesting but markets are driven by people's behaviour and if "quantified news" can help us understand this better it has to help in avoiding (some!) of the future problems to be faced in the market.

Posted by Brian Sentance | 12 November 2009 | 12:06 am

Truly "Open" Bloomberg?

Interesting couple of articles from Inside Reference Data and Inside Market Data. The first is on Bloomberg making its codes freely available to all from its website http://bsym.bloomberg.com - given past standards-based attempts like ISINs falling short of providing the industry with unique and useful security IDs this looks to be a welcome addition. This seems to be a publicity "win" for Bloomberg, especially given rival Thomson Reuters has recently got some indifferent publicity with the EU over RIC licensing (see article). No prizes for anyone who thinks that Thomson Reuters will not respond in some way with regard to RIC usage, maybe giving us two working proprietary standards that go "open" - at least everyone would then be matching up Bloomberg Tickers and Reuters RICs in public rather behind closed doors - and maybe a good opportunity for a Wiki site to do the matching up?

The second relates to Bloomberg providing a open-source data distribution system called "The Platform", I presume as less expensive alternative to Reuters RMDS. Meanwhile Reuters is busying itself with the plans for its competitor to the Open Bloomberg terminal with "Project Utah". Obviously Bloomberg is comparatively unproven with regard to systems provision so this is a big change and will be very interesting to watch - from a technology point of view but also culturally since can Bloomberg turn away from thinking in "Terminals" all of the time?

Posted by Brian Sentance | 3 November 2009 | 3:52 pm

Integrated Data and Analytics Management

Xenomorph was one of the sponsors on the “Integrated Data Management” webcast last week, hosted by Inside Reference Data (audio recording available here). There were a number of interesting questions that arose from the Webinar.

One fundamental although somewhat academic question was "What is Integrated Data Management?". Certainly everyone seemed convinced that there would be less "Enterprise Data Management" (EDM) projects in future, given the expense, scope and scale of such projects. The concensus was that whilst the need for data management was better under stood across all financial institutions, data management projects would be bitten off in more manageable chunks by asset type, business function or division (so are silos back in fashion I ask myself?!). Coming back to the original question, I guess my slant on Integrated Data Management is that we are seeing more and more data management projects that have an integrated reference data and market data elements to them, primarily driven by the need to sort out data quality/completeness/depth for use within risk management (in light of the financial crisis).

Related to risk management, a topic I pushed was that given the origins of data management for STP/back office, and given the interest in low latency tick data management/analyis in the front office, there seems to be a market gap (particularly in the US?) on how to manage data such as IR/credit curves, volatility surfaces and other derived data sets. These data sets seem to fall into the gap between what is thought of as market data (primarily just prices) and what is reference data (IDs and terms & conditions). This is another area where a more integrated approach to data management would be beneficial, particularly in making all these datasets available for risk management.

Coming back to a "hobby-horse" of mine, then I also raised the issue that whilst it is fine to be doing great data management (high quality, complete datasets etc) what is the point if all of your data is ignored by the front office and Excel is used to download the data traders and risk managers need from Open Bloomberg. I think the management of unstructured data (spreadsheets, word docs etc) needs to be elevated as an issue since this (unfortunately?) is where most data resides currently, despite what we data management professionals like to think.

I also think that the principles of good data management (centralisation, quality and transparency) could apply to other things and not just raw "data", but what about centralised pricing and valuation, centralised curves and centralised scenarios for risk? Again what is the point of doing good data management if the ultimate "information" (e.g. a valuation) is done using poor quality data, with a complete lack of transparency over the data and model used.

A good question was asked about models, which was that given pricing models and their weaknesses have formed some part of the recent crisis, do we need more complex models. On having a few conversations about this and thought about it some more, then some would say it is complexity that got us into the crisis so this is the last thing we need. My view is that we do not necessarily need more complex pricing models and valuation techniques, but we certainly need more robust ones which does not necessarily imply more complexity. Coming back to a point raised by David Rowe previously, then I think all quants and risk managers should think about a "second means of valuation" for all the theoretical models they use, and that hedgeability (see recent post on pricing model validation) seems to be the common theme in producing more robust pricing models.

Posted by Brian Sentance | 21 October 2009 | 10:32 am

Pricing Model Validation: Mitigating Model Risk

I managed to catch some of the day yesterday at the "Pricing Model Validation: Mitigating Model Risk" conference. I thought it would be worthwhile going along since firstly the past 12-18 months have made model risk very topical (take a look at previous posts from Riskminds, the Modeller's Manifesto and Wilmott/Rowe).

Secondly more of our clients are looking at managing and centralising pricing models/curve calculators in addition to just managing the underlying data (see this Insight Investment client case study for a recent public example). I am calling this "Analytics Management" which is the business-focussed technology stack that combines pricing models/calculators/analytics with all of the "Data Management" underneath. But enough of my thinly-veiled positioning statements...and on with some of the (hopefully) useful content from the conference outlined below - maybe scan the headings in bold below for those talks of interest but I would particularly recommend the ones by Tanguy Dehapiot and Yuyal Millo...

Model Risk 2009 defining and forecasting. First speaker was Professor Phillip Sibbertsen of the University of Hannover on defining and measuring model risk. Phillip started by saying that "Model Risk" was a new category of risk within the confines of "Operational Risk", and that operational risk as defined by the regulators does not yet currently include the "model risk" of market risk and credit risk, nor the "model risk" of the operational risk model itself. (I am sure I could write that up better!...). Phillip put forward that model risk is not formally a "risk" since it has no probability distribution and that he suggested it should be thought of as "model uncertainty". He also clarified that model risk applies both at the large, portfolio scale (e.g. choice of VAR model etc) and at the smaller, instrument level scale (i.e. pricing of derivatives).

Additionally in terms of measuring model risk then he excluded human failure from model risk measurement since in his view this was difficult to quantify - this approach did not meet with the approval of some of the audience were questioning how this could be excluded from a practical point of view. Phillip's colleague, Corinna Luedtke, then presented some work they had done on calibrating different GARCH models to observed data and showing how even a poor model could produce reasonable forecasts of risk if the time period was short. The work was interesting but again the audience highlighted that the human choice (failure?) in choosing the set of models to try was part of "model risk" and should not be excluded from the definition of model risk.

Is a model accurate? Testing the implementation of a model. Second speaker was David Chevance, Head of Equity & FX Model Validation at Dresdner Kleinwort. David outlined the different sorts of model risk: mathematical errors, missing risk factors, divergence from industry practice, model inconsistencies and implementation risk. He then outlined the sources of these risks: bugs, approximations, numerical precision, numerical boundaries and limitations on numerical methods (e.g. Sobol numbers in high dimension monte-carlo simulations).

David said a key area to start with in validating a model implementation was the front-office documentation of the product, its inputs and payoffs, its pricing model but also details of calibration methods used/needed etc. He made the point here that the documentation can sometimes specify just the deal, but sometimes can express the pricing methodology and pricing parameters. The emphasis was on completeness, accuracy and making use of all of the information available in the documentation. Obviously the ability to review the code used to implement the model was also necessary.

He discussed the trade-offs between a simple validation approach in terms of speed and efficiency of resources against the more time-consuming, resource hungry but more accurate approach of full replication of the model. He also suggested that in choosing a method of validation it was important to balance resource demands against what is actually being validated: payoffs from a single trade, a type of pricing model or a family of financial products. Desired accuracy of the validation was also important, given the trade-off between accuracy and effort, and the fact that small bugs are much more common than large.He finally discussed model version control, the necessary discipline of documenting changes and regression tests for new models, and the regular cycle of model review. Overall it was an interesting talk with a good practical focus.

Practical aspects of valuation model control process. One of the most entertaining and interesting speakers of the day was Tanguy Dehapiot, Head of Validation and Valuation, Group Risk Management at BNP Paribas. He started by referring to a few documents "Supervisory guidance for assessing banks’ financial instrument fair value practices", April 2009 (BCBS 153) which was then implemented within “Enhancement to the Basel II framework” (BCBS 157). The first part of his presentation was around these documents and what the regulators expect to be in place, so I guess the best approach is to read them (the BCBS 153 document content is only 12 pages long, quite short for a regulator!)

Tanguy pointed out that in his view "Mark to Market" and "Mark to Model" are often misleading as both are often required. He prefers the term "Valuation Methodology". He proposed four valuation modes: Direct Price Quotation, Use of Similar Instruments, Risk Replication, Expected Uncertain Cashflows (NPV) and categorised a useful hierarchy/matrix of which financial products fit into which valuation mode and for what purposes. Within model risk, he split off judgemental errors (choice of model etc) as part of market risk and credit risk and operational errors (model implementation and coding) as more definable and avoidable parts of operational risk.

He had some interesting slants on data, saying that he had been surprised that even getting all of the static data necessary to price simpler instruments like bonds had proven difficult. He outlined how model parameters are often stored across a variety of systems (curve definitions in one place, pricing methodology somewhere else) implying to me that this is sometimes difficult to pull together and needs some centralisation to improve transparency around this.

His opinion on market parameters (both observed prices and derived data such as implied volatility surfaces) were often stored in a larger central database but warned that this market parameter database needs to be reviewed as part of the model validation process since some of its data is derived (i.e. calculated, maybe using a model!) and as such should not be taken as perfect for all time and for all purposes. He said that it was important to categorise the origin of data and suggested the following types:

Quoted on an active exchange
Actual private transaction in an active market
Tradable broker quotes
Consensus prices from market makers
Non-binding indicative prices from market makers
Counterparty valuation, collateral valuation
Actual transactions in inactive market

Tanguy proposed that there should a valuation matrix for each instrument, where there might a different valuation methodology used for end of day valuation verses intraday, for risk or for trading, for pricing individually or within a portfolio reval. I guess here the rational is appropriateness, efficiency and transparency about what needs to used when. He also added that he disliked the term "Model Validation" since it seemed to imply that a model was "valid" and preferred "Model Approval" to cover the decision to use a model and "Model Review" to cover model analysis. He said he found managing the "stock" of existing models (and keeping up with when to review them) more difficult than managing the "flow" of new models and products.

Overall Tanguy was a very interesting and funny speaker with lots of practical insights and a fair amount of opinion thrown in, which is always good in my view.

The usefulness of inaccurate models: Financial risk management "in the wild". This talk was given by Dr Yuval Millo of the London School of Economics and he focussed on the evolution of the use of the Black Scholes Merton (B-S-M) model at the CBOE and how the model came to be the means by which the whole options market "communicated". Yuyal is a social scientist and prefaced his talk by stating that "Social Sciences are good at predicting the past"

First thing I didn't know (amongst the many things I do not know...) is that the B-S model was not published until a couple of weeks after the CBOE started trading stock options in April1973. Yuyal said that initially the B-S-M derived prices were not accurate at all (around 25% off the market price on CBOE) and that the model was based on assumptions that plainly were not the case on the exchange (only calls available, no short selling, no continuous trading). The model was used by local Chicago trading firms and the story goes that Fischer Black sold large paper "sheets" of option pricing matrices to these traders (there being no calculators/PCs/mobiles around at the time).

As the markets developed, larger East Coast banks entered the market with stocks being held and traded in New York and options being traded in Chicago, so trading became geographically dispersed. This started the need for "early morning meetings" to discuss the market and the B-S-M model and its parameters became the "lingua franca" or means of communication of options market participants.

He described the first years of the Options Clearing Corporation (OCC) which was set up to ensure that the financial obligations of options and buyers were met. Around 1979-80 the OCC worked overnight to calculate margin requirements, based on the (now?) arcane idea that different margin amounts should be associated with different option strategies (straddles, butterflies etc) and the job of the OCC was to take a portfolio of Option and optimise which combination of strategies would minimise the margin required for the whole portfolio. He said that there were disputes between traders and the OCC around margin levels and difficulties for the SEC with updating their Net Capital Rules as each new option strategy was created. Eventually, the OCC adopted the B-S-M model and implied volatility as the means of calculating margin against market value which enabled them to move away from the operational difficulty of strategy optimisation.

So the B-S-M became the way in which traders communicated about the market but also the model became vital operationally within clearing for the market. By 1987 B-S-M had become the de-facto standard for the market, with the model driving the market in turn driving use of the model. During the Oct '87 crash the model proved to be very innaccurate but the use of the model did not diminish - maybe pschologically the market participants needed a model (even a wrong model) to make communication easier.

I found this talk very interesting and members of the audience asked whether any similar analysis was going to be done on the Gaussian Copula model used to price CDOs. Yuyal said that one of his colleagues was undertaking this research currently. Given that he seemed to be very positive about the use of the B-S-M model within options markets I asked whether he had any opinions on Taleb's criticism of fiancial engineers and modelling. Yuyal said that he and Nassim were friends and agreed to disagree on certain topics...

Stress testing modelling parameters. Next up was Peirpaolo Montana, Head of Model Validation at West LB. Having joined the finance industry out of a career in mathematics and then at a regulator, Pierpaulo began by saying that back in the heady days of 2004 the banks thought that their own risk management systems and practices were well ahead of the regulators. He said that in light of the crisis this proved not to be the case but he now feels that this is now more evenly balanced (not sure I would agree, still lots of catchin to do for some institutions I would suggest).

He said that whilst regulators require the validation of risk models and pricing models, and that stress testing of a portfolio is required, that the stress testing of a pricing model is not a requirement and has received much less attention and in his view was not done to much degree before 2007. His point here was that pricing models should work under stress too, otherwise they are a weak foundation for building other risk measures such as stressed VAR.

Whilst focussing on pricing models, he mentioned that risk models also need to be carefully chosen and appropriate to the institution and the types of trading activities it undertakes. As an example he put forward that a simple VAR calculator might be appropriate for a long only equity fund but completely innappropriate for a relative value portfolio.

He said that stress testing had recently received much more attention as a risk management tool and cited the BIS document "Revisions to the Basel II market risk framework" where stressed VAR is introduced as part of the regulatory capital charge calculation. He also mentioned that in order to avoid "standard model" treatment of complex securitised products an institution must be able to demonstrate that its VAR model can cope with these products under times of market stress.

Pierpaulo then described the stress testing of base correlation in CDO pricing, and how even moving the base correlation from its usual level of 70% to 99% would not have predicted the valuations observed in the recent crisis. In this way he says that stress testing of models can detect implementation problems and some model weaknesses, but it cannot assist in coping with structural breaks in the market. He also discussed how the B-S-M model is used everywhere (even places it should not really be valid for) since it is a robust model based on the no-arbitrage hypothesis - in contrast the CDO base correlation and other models are not so robust since they are not arbitrage free.

(end of post!)

Posted by Brian Sentance | 18 September 2009 | 5:30 pm

Heavyweight Data Management...

...I am very concerned that I have previously missed an important requirement for data management solutions - a heavweight one judging by this great discussion on one of the Microsoft forums.

Posted by Brian Sentance | 17 July 2009 | 8:17 am

Microsoft CEP Surfaces as "Orinoco"

Seems like Microsoft have now gone public on the Microsoft TechEd site that they have a Complex Event Processing (CEP) engine that will be coming to market shortly (see MagmaSystems blog post ). One of my colleagues Mark Woodgate attended a briefing event at Microsoft for this technology back in February this year - here's an extract from some internal notes that Mark made back then:

"Microsoft CEP is very similar to StreamBase conceptually (and not unsurprisingly), in the sense that there are adapters and streams and how you merge and split them via some kind of query language is the same. However, StreamBase uses the StreamSQL which as we have seen is SQL-like in syntax but Microsoft CEP uses LINQ and .NET and although conceptually it is doing the same thing, it does not look the same. StreamBase’s argument was you can be an SQL programmer to use it and don’t need lower-level like .NET; however, it’s not SQL really as it has all these ‘extensions’ you have to learn so using .NET might look more tricky but in fact it makes sense. They don’t have a sexy GUI yet for designing CEP applications like StreamBase but it will be done in Visual Studio 2008.

Currently, you build various assemblies (I/O adapters, queries and functions) and then bolt them all together, called ‘binding’ by command line tool. You then deploy the application onto one or more machines using another tool so it’s a manual process right now. They are aware this needs to be made easier and more visual. They are allowing other libraries to be bolted in via the various SDKs so it’s pretty open and flexible. It works well with HPC and clusters/grids (or so they say) and of course can be used with SQL Server. The CEP engine also has a web interface based on SOAP so at least non-Windows based systems can talk to it"

The release of this technology will be an interesting addition to the CEP market and to the Microsoft technology stack in general. Assuming performance is at credible levels (i.e. not necessarily leading but not appalling either) it will certainly bring both technical and commercial pressure to bare on existing CEP vendors (see earlier post on Aleri/Coral8) and has the potential to broaden the usage of CEP. Obviously Linux-Lovers (sorry, I didn't mean to be personal...) will not agree with this, but Microsoft is putting together an interesting stack of technology when you see this CEP engine, Microsoft HPC and Microsoft Velocity coming together under .NET.

Posted by Brian Sentance | 14 May 2009 | 5:13 pm

Data Quality and the Future of Risk

A new survey from the Economist Intelligence Unit (sponsored by SAS) of over 300 financial institutions world-side has put data quality and availability as a key issue to be resolved if risk management is to be fit for purpose following the financial crisis:

"Culture, expertise and data are weak points in current risk management"

A summary of the survey report is available here.

Posted by Brian Sentance | 8 May 2009 | 10:31 am

High Performance Spreadsheets

Another article about the operational risk generated by the usage of spreadsheets within the financial markets (see earlier posts), appeared in the April issue of Waters Magazine.

The articles highlights how spreadsheets are largely used within financial institutions and suggests that the current regulation requirements for more transparency and ad-hoc risk management might push the proliferation of spreadsheets even further. The articles also refers to the progress and improvements made by Microsoft in recent versions of Excel to increase the security of spreadsheets.

Xenomorph has worked closely with Microsoft on hosting its time series database within SQL Server 2008. The case study we have written together describes how SQL Server 2008 offers integration within Office Excel 2007 so that whilst the spreadsheet is still the end-user viewing tool, operational risk is reduced by engaging Excel 2007 as an analytics and reporting tool and not as a mean of storing data.

Our TimeScape solution offers more than 700 easy to use add-in functions to Office Excel 2007 and we are currently working on the use of Excel Services, part of Microsoft Office Share Point Server 2007, to further enhance the centralized approach to spreadsheet.

If you are interested in how Xenomorph solves the problem of spreadsheet management, then take a look at our (newly updated) website. Here we explain how to solve the problem and how Xenomorph Spreadsheet Inside technology can bring unstructured spreadsheet data and complex calculation within a centralized data management system, increasing transparency and reducing operational risk.

Posted by Brian Sentance | 8 April 2009 | 2:35 pm

Data management, derivative analytics and the spreadsheet

Interesting article out doing the rounds on the newswires announcing a forthcoming report called "The Enterprise Spreadsheet: Pushing towards Transparency" by the analyst firm the Tabb Group. It is great to see an analyst firm acknowledging the importance of spreadsheets within the markets, particularly in the area of combining data and analytics together in OTC derivatives management (see earlier post).

Adam Sussman of the Tabb Group reckons that despite its shortcomings, Excel is a valuable tool: “Spreadsheets, either alone or in conjunction with other components, can meet the same requirements as a business application.” In this he seems to be agreeing with the UK Regulator the FSA, who have been recently advocating that spreadsheets and spreadsheet data needs actively managing as an institutional resource. The findings of the Tabb Group on management also seem to echo a recent report called "Buy-Side Data Management in a Changing Landscape" done by Lepus for Asset Control (registered link to report here).

Spreadsheets are a great tool and fulfil a real need in the market to pull together pricing models and data quickly, easily and with a timeframe that is meaningful to the business (see earlier post for some work by Xenomorph in this area). Spreadsheets are a big problem to manage, but they are also the symptom of failings in core systems that are not able to rapidly support new instrument types and pricing models. An institution that ignores analytics, spreadsheets and spreadsheet data within any EDM transparency initiative has already failed before it begins, and so to paraphrase the author Aldous Huxley:

"Spreadsheets do not cease to contain data because they are ignored."

Posted by Brian Sentance | 13 February 2009 | 2:54 pm

EDM Council on the Meaning of Data

The EDM Council has today released its first draft of a Semantics Repository for review by the industry. Its intent is to establish a common meaning for terms used in financial markets and data management in particular. How it is meant to be used and how it fits with other standards such as FpML and MDDL is briefly outlined by the following article.

As already mentioned in our earlier post, it would be great for the industry if this time of crisis provides the need and motivation to get standards in place. I guess we are all fighting against the profound Tannebaum quote of "the nice thing about standards is that there are so many of them to choose from".

I wish the EDM Council every success with this effort, although the irony that we vendors and users of centralised data management systems and proponents of the single "Golden Copy", need to do better ourselves on establishing centralised industry standards for data. As I say to my five year old son "do as I say, not as I do"...

Posted by Brian Sentance | 28 January 2009 | 6:44 pm

CEP in 2009

Interesting predictions for complex event processing (CEP) in 2009 (click here for link) - sounds like some form of reality is appearing in this area of the market, accelerated by the current financial crisis. Entry of bigger players and usage of LINQ in CEP will be interesting too.

Posted by Brian Sentance | 25 January 2009 | 4:02 pm

Happy Birthday Spreadsheet!

Article on PCMag saying that the spreadsheet is 30 years old. Whilst wishing it a happy birthday, the author, John C. Dvorak, has a good rant about how spreadsheets have been the major weapon in the rise to power of the accountant in business.

Good job he did not spend too much time looking at their usage in financial markets or else his rant would have been much longer, given past issues with spreadsheets in financial markets. The spreadsheet (which means Excel at the moment) is a great tool that is:

a calculator;
a report writer;
a database

In my view it is the latter usage of desktop spreadsheets to store data where the problems mainly reside, not its usage as an analysis tool. Faced with inflexible trading and risk management systems that do not allow instrument and trade data to be represented quickly or correctly, it is unsurprising that traders, portfolio managers and risk managers resort to spreadsheets as the "pressure relief valve" for their business activities.

Delivering systems that can support both complex and non-standard instruments and trades in a transparent manner should be a focus in a world where that the lack of transparency over credit derivative pricing has been such an issue. The inappropriate usage of spreadsheets is a very small part of the current problems we are experiencing in the markets, but addressing this would be a positive step in creating a data management foundation that encompasses all data used by a financial institution, not just that data that is easy for software vendors to represent in their systems.

Anyway, enough of my spreadsheet hobby-horse, for some light relief and to celebrate 30 years of summing rows and columns, take a look at the Eusprig web site for a list of the most notable spreadsheet failures.

Posted by Brian Sentance | 15 January 2009 | 6:42 am

Libor no more...

Following the ongoing story of Libor diverging from the OIS rate (see earlier post), Risk magazine reports that Libor risks losing its place as a funding benchmark. Spreads against the OIS have tightened recently (see recent article in the FT), but Mustafa Chowdhury, head of US interest rate research at Deutsche Bank in New York, says that Libor is becoming less relevant as a benchmark due to banks accessing other sources of funding such as Federal Reserve Funds.

Time to change all of those benchmark yield curves across the entire institution and understand all of the pricing differences? Ouch! Maybe wait a while yet...

Posted by Brian Sentance | 14 January 2009 | 2:38 pm

MiFID Market Data Deterioration

Members of the Investment Management Association (IMA) are not happy about the quality of market data following from the first year of MiFID being with us. According to an article in the FT this morning, a survery of IMA members has voiced concern over the fragmentation of trading venues leading to a deterioration in the quality of market data available.

Most institutions seem positive about the benefits of competition that multiple trading venues such as Chi-X, Turquoise and BATS bring, but IMA members would like to see a centralised venue where prices are consolidated and made available to the market (for free?), similar to the "consolidated tape" available for US markets.

Sounds like the institutional need for centralised data management across different departments and systems is also becoming apparent at a market and exchange level following MiFID. Multiple trading venues and decentralisation is necessary to bring the benefits of competition, but these benefits unsurprisingly do not come without costs or issues (see earlier post).

Posted by Brian Sentance | 25 November 2008 | 11:25 am

Unstructured Data Management Anyone?

Good example on Finextra this week of another spreadsheet debacle, this time involving Barclays and Lehmans (see article).

Not sure when the financial markets will get serious about managing unstructured data and spreadsheet usage. We can all apply Enterprise Data Management until the cows come home, but if the "real" business is being managed in spreadsheets then really why bother with grand aims like EDM?

Even the regulators are not totally against the usage of spreadsheets (see presentation), then simply want unstructured data to be managed properly as part of an institution's formal processes and not ignored until the inevitable problems arise...

Posted by Brian Sentance | 20 October 2008 | 6:05 am

Transparency for troubled times

I came across this pair of quotes on a google search, bringing data management into the context of the current financial crisis:

"Where is the wisdom? Lost in the knowledge. Where is the knowledge? Lost in the information." - T.S. Eliot

"Where is the information? Lost in the data. Where is the data? Lost in the ******* database." - JoeCelko

Here's to hoping that wisdom is not in short supply at the moment...

Posted by Brian Sentance | 6 October 2008 | 7:00 am

Here today. Where tomorrow?

These may be the words on the lips of many bankers today, as they survey the continuing turmoil in global financial markets. In fact, this was the incredibly apposite tagline on a recent magazine advertisement for a major bank which (maybe unsurprisingly) was subsequently nationalised.

In the fluid (many would say “bloody”) landscape of financial services, with the next merger or acquisition just around the corner, it means that now, more than ever, data integration is a growing challenge. Accompanying this activity is the ever-growing need for consistency, accuracy, transparency and control of both the data itself and the movement of that data.

Data architecture itself is an evolving discipline and one approach worth looking at is data federation – deftly described in an article by Dain Hansen. Basically, the approach is to leave the data where it is but aggregate it into a single view, available as a service to your applications. It is an approach that Xenomorph has advocated for many years, going back to our founding days in the mid-90s, with the normalized database driver approach implemented in our Connectivity Services.

Hansen’s article explains both the advantages (simplicity, no need to copy or synchronize) and the disadvantages (performance) of this approach, and argues for a solution that incorporates both federation and consolidation of data. He shows that it is possible to architect a system that will provide consistency and control as well as agility.

It’s difficult to say whether better data management would have assisted the world’s banks in avoiding their current troubles, but greater transparency of where exactly their exposures lay would certainly have helped.

Posted by David Winson | 1 October 2008 | 6:28 pm

Real-time holidays...

...on a happier note than real-time death data (see earlier post) then holiday data is also real-time - a link with statistics from one of our hedge fund contacts on holiday data:

http://www.financialcalendar.com/freestuff/getreal.htm

Posted by Brian Sentance | 2 June 2008 | 6:29 pm

Vhayu and Streambase - positioning clarified?

Partner announcement on Finextra with Vhayu and Streambase coming together:

http://www.finextra.com/fullpr.asp?id=21477

Defining what vendors mean by a "Data Management System" is difficult enough for clients, but in the area of the somewhat fuzzy technology definitions around automated trading it is interesting to see Streambase clarify their offering around CEP (and not database too, which was one of their first messages around bringing real-time and historic data together), and that Vhayu seems to be emphasising its tick database capabilities (and de-emphasising its original perception in the market as a CEP vendor).

Posted by Brian Sentance | 21 May 2008 | 11:29 am

Streaming Blue Genes...

The supercomputer continues to make a come-back - just up on Finextra with TD Bank testing IBM's Blue Gene supercomputers to amalgamate and analyse real-time structured and unstructured data:

http://www.finextra.com/fullstory.asp?id=18293

Posted by Brian Sentance | 2 April 2008 | 4:11 pm

Far away from low latency...

...is the OTC derivatives market - still a lot to be done according to Finextra report and letter from the Fed:

http://www.finextra.com/fullstory.asp?id=18285

Posted by Brian Sentance | 1 April 2008 | 4:23 pm

Time Series inside SQL Server

Case study of some of the work we have been doing with Microsoft on hosting our time series storage inside SQL Server has just gone up on their site at:

http://www.microsoft.com/casestudies/casestudy.aspx?casestudyid=4000001637

Posted by Brian Sentance | 19 March 2008 | 1:24 pm

Higher quality data from the front office?

Sungard had a good event on Thursday night, with four risk managers taking the stage for a "thought leadership" seminar entitled "Regulatory Impact of Market Events" (if the advert is still around on their site, see http://www.sungard.com/ADAPTIV/default.aspx?id=4678&formAction=takeit&formid=48)

The Dresdner risk manager (Ted Macdonald, good speaker) was emphasising that data quality is a real issue for risk management, and that all participants thought that risk managers should spend more time on risk and less on validating/cleaning data (no great surprises there then but interesting to hear it validated again as an issue).

He suggested that more pressure should be put on the front office to get data right first time (as opposed to leaving everyone else to sort out the mess!), even going so far as to suggest that charging the front office for each wrongly-booked trade in the trading and risk management systems - not sure how that would go down with the trading desks, but sounds a good approach if you could agree (and unambiguously measure) these mistakes!

Seems like transfer-costing is becoming a re-occurring theme - also recently mentioned by a grid computing specialist from Credit Suisse about "metering" each desk for the amount of compute power used...anyone retraining as a management accountant out there? - sounds like the banks will be hiring soon!

Posted by Brian Sentance | 17 March 2008 | 3:08 pm

Best Instrument Data Model?

Some positive feedback about instrument and market data model within our data management system TimeScape:

http://mdavey.wordpress.com/2008/03/01/xenomorph-best-data-modelling/

Posted by Brian Sentance | 7 March 2008 | 1:18 pm

« Back to recent posts

Contact us if you have comments. All rights reserved. Trademarks, copyright and legal. Whole site ©1995-2010 Xenomorph Software Ltd. Registered in England and Wales, Reg no: 03235432, Reg at: Waverly House, 7-12 Noel St, London, W1F 8GQ. VAT no: 672584016 - sitemap

Categories

Posts categorized "Data Management"