Xenomorph Blog - Data Management

Posts categorized "Data Management"

PRMIA - Operational Risk, Big Data and Human Behaviour

I attended Challenges and Innovations in Operational Risk Management event last night which was surprisingly interesting. I say surprising since I must admit to some prejudice against learning about operational risk, which has for me the unfortunate historical reputation of being on the dull side.

Definition of Operational Risk

Michael Duffy (IBM GRC Strategy Leader, Ex-CEO of OpenPages) was asked by the moderator to define Operational Risk. Michael answered that he assumed that most folks attending already knew the definition (fair comment, the auditorium was full of risk managers...), but he sees it in practice as the definition of policy, the controls to enforce the policies and ongoing monitoring of the performance of the controls. Michael suggestion that many where looking to move the scope and remit of Operational Risk into business performance improvement, but clients are not there yet on this more advanced aspect.

Vick Panwar (Financial Services Industry Lead, SAS) added that Operational Risk was there to mitigate the risks for those unexpected future events (getting into the territory of Dick Cheney's Unknown Unknowns which I never tire of, particularly after a glass of wine).

Rajeev Lakra (Director Operational Risk Management, GE Treasury) took his definition from Basel II of Operational Risk as risk of loss resulting from inadequate or failed internal processes, people and systems, or from external events. Coming from GE, he said that he thought of best practice Operational Risk as similar to another GE initiative in the use of Six Sigma for improving process management. Raj said that his operational risks were mainly concerned with trade execution so covering data quality/errors, human error and settlement errors.

Beyond Box Ticking for Operational Risk

Raj said that Operational Risk is treated seriously at GE with the Head of Operational Risk reporting into the CRO and leaders of Operational Risk in each business division.

Michael suggested that the "regulators force us to do it" motive for Operational Risk had reduced given some of the operational failures during the financial crisis and recent "rogue trader" events, with the majority of institutions post-2008 having created risk committees at the "C" level and being so much more aware of tail events and the reputational damage that can damage shareholder value.

Vik said that Operational Risk is concerned primarily with "tail events" which by definition are not limited in size and therefore should be treated seriously. Pragmatically, he suggested that "the regulators need it" should be used as an excuse if there was no other way to get people to pay attention, but getting them to understand the importance of it was far more powerful.

The "What's in it for you" Approach to Operational Risk

Raj emphasised that it was possible to emphasise the benefits of operational risk to people in their everyday jobs, explaining to operators/managers that if they get frustated with failures/problems in the working day, then wouldn't it be great if these problems/losses were recorded so that they could justify a process change to senior management. He emphasised that this was a big cultural challange at GE.

Michael suggested that his clients in financial markets had gone through risk assessment, controls and recording of losses, but had not yet progressed to the use of Operational Risk to improve business performance.

Duplication of Effort

A key thing that all the panelists discussed was the overlap at many organisations between Operational Risk, Audit and Compliance. The said that the testing of the controls used for each had much in overlap, but was not based on a common nomenclature nor on common systems. For instance Vik pointed out that many of the tests on controls in Sarbanes-Oxley compliance were re-usable in an Operational Risk context, but that this was not yet happening. Vik said that this pointed to the need for comprehensive GRC platform rather than many siloed platforms.

Michael said that regulators want an integrated view, but no institution has an integrated nomenclature as yet. He recounted that one client sent 12 different control tests to branches that needed to be filled in for head office, which was a waste of resources and confusing/demotivating for staff. Raj said that the integration of Audit and Operational Risk at GE had proved to be a very difficult process. All agreed that senior management need to get involved and that a 5 year vision of how things should be incrementally integrated needs to be put in place.

Audience Questions:

Is business process risk different to business product risk? Michael said that Operational Risk certainly does and should cover both internal process and also the risks produced by the introduction of a new financial product for instance (is it well understood for instance, do clients understand what they are being sold?). He added that Operational Risk encompassed both the quantitative (statistical number of failures for instance) and the qualitative for which statistics were either not available (or not relevant to the risk).

Are there any surrogate measures for Operational Risk? Here a member of the audience was relaying senior management comments and frustration over the stereotyped red/amber/green traffic lights approach to reporting on operational risk. Michael mentioned the Operational Riskdata eXchange Association (ORX) where a number of financial institutions anonymously share operational risk loss data with a view to using this data to build better models and measures of operational risk. Apparently this has been going on since 2003 and the participants already have a shared taxonomy for Operational Risk. (my only comment on having a single measure for "operational riskiness" is that do you really want a "single number" approach to make things simple for C-level managers to understand, or should the C-levels be willing to understand more of the detail behind the number?)

Is "Rogue Trading" Operational Risk? Michael said that it definitely was, and that obviously each institution must control and monitor its trading policies to ensure they were being followed. The panel proposed that Operational Risk applied to trading activity could be a good application of "Big Data" (much hyped by industry journalists lately) to understand typical trading patterns and understand unusual trading patterns and behaviours. (Outside of bulk tick-data analysis this is one of the first sensible applications of Big Data so far that I have heard suggested so far given how much journalists seem to be in love with the "bigness" of it all without any business context to why you actually would invest in it...sorry, mini-rant there for a moment...)

Summary

Good event with an interesting panel, the GE speaker had lots of practical insight and the vendor speakers were knowledgeable without towing the marketing line too much. Operational Risk seems to be growing up in its linkage into and across market, credit and liquidity risk. The panel agreed however that it was very early days for the discipline and a lot more needs to be done.

Given the role of human behaviour in all aspects of the recent financial crisis, then in my view Operational Risk has a lot to offer but also a lot to learn, not least in that I think it should market itself more agressively along the lines of being the field of risk management that encompasses the study and understanding of human behaviour. Maybe there is a new career path looming for anthropologists in financial risk management...

Posted by Brian Sentance | 27 January 2012 | 11:30 pm

Latest from the EDM Council

Click here for an executive summary of what the EDM Council is up to on regulation, LEI and the Semantics Repository etc. Due credit to the Council for getting Bloomberg on board - sounds increasingly like Bloomberg may have decided to treat the topic seriously as opposed to assuming having a terminal solves everything.

Posted by Brian Sentance | 13 January 2012 | 4:24 pm

PRMIA - From Risk Measurement to Risk Management by Samuel Won

I attended the PRMIA event last night "Risk Year in Review" at Moody's New York offices. It was a good event, but by far the most interesting topic of the evening for me was from Samuel Won, who gave a talk about some of the best and most innovative risk management techniques being used in the market today. Sam said that he was inspired to do this after reading the book "The Information" by James Gleik about the history of information and its current exponential growth. Below are some of the notes I took on Sam's talk, please accept my apologies in advance for any errors but hopefully the main themes are accurate.

Early '80s ALM - Sam gave some context to risk management as a profession through his own personal experiences. He started work in the early 80's at a supra-regional bank, managing interest rate risk on a long portfolio of mortgages. These were the days before the role of "risk manager" was formally defined, and really revolved around Asset and Liability Management (ALM).

Savings and Loans Crisis - Sam then changed roles and had some first hand experience in sorting out the Savings and Loans crisis of the mid '80s. In this role he become more experienced with products such as mortgage backed securities, and more familiar with some of the more data intensive processes needed to manage such products in order to account for such factors such as prepayment risk, convexity and cashflow mapping.

The Front Office of the '90s - In the '90s he worked in the front office at a couple of tier one investment banks, where the role was more of optimal allocation of available balance sheet rather than "risk management" in the traditional sense. In order to do this better, Sam approached the head of trading for budget to improve and systemise this balance sheet allocation but was questioned as to why he needed budget when the central Risk Control department had a large staff and large budget already.

Eventually, he successfully argued the case that Risk Control were involved in risk measurement and control, whereas what he wanted to implement was active decision support to improve P&L and reduce risk. He was given a total budget of just $5M (small for a big bank) and told to get on with it. These two themes of implementing active decision support (not just risk measurement) and have a profit motive driving better risk management ran through the rest of his talk.

A Datawarehouse for End-Users Too - With a small team and a small budget, Sam made use of postgraduate students to leverage what his team could develop. They had seen that (at the time) getting systems talking to each other was costly and unproductive, and decided as a result to implement a datawarehouse for the front office, implementing data normalisation and data scrubbing, with data dashboard over the top that was easy enough for business users to do data mining. Sam made the point that useability was key in allowing the business people to extract full value from the solution.

Sam said that the techniques used by his team and the developers were not necessarily that new, things like regression and correlation analysis were used at first. These were used to establish key variables/factors, with a view to establish key risk and investment triggers in as near to real-time as possible. The expense of all of this development work was justified through its effects on P&L which given its success resulting in more funding from the business.

Poor Sell-Side Risk Innovation - Sam has seen the most innovative risk techniques being used on the buy-side and was disappointed by the lack of innovation in risk management at the banks. He listed the following sell-side problems for risk innovation:

politically driven requirements, not economically driven
arbitrary increases in capital levels required is not a rigorous approach
no need for decision analysis with risk processes
just passing a test mentality
just do the marginal work needed to meet the new rules
no P&L justification driving risk management

Features of Innovative Approaches - Sam said that he had noted a few key features of some of the initiatives he admired at some of the asset managers:

Based on a sophisticated data warehouse (not usually Oracle or Sybase, but Microsoft and other databases used - maybe driven by ease of use or cost maybe?)
Traders/Portfolio Managers are the people using the system and implementing it, not the technical staff.
Dedicated teams within the trading division to support this, so not relying on central data team.

A Forward-Looking Risk Model Example - The typical output from such decision analysis systems he found was in the form of scenarios for users to consider. A specific example was a portfolio manager involved in event-driven long-short equity strategies around mergers and acquisitions. The manager is interested in the risk that a particular deal breaks, and in this case techniques such as Value at Risk (VaR) do not work, since the arbitrage usually requires going long the company being acquired and short the acquiror (VaR would indicate little risk in this long-short case). The manager implemented a forward looking model that was based on information relevant to the deal in question plus information from similar historic deals. The probabilities used in the model where gathered from a range of sources, and techniques such as triangulation where used to verify the probabilities. Sam views that forward-looking models to assist in decision support are real risk management, as opposed to the backward-looking risk measurement models implemented at banks to support regulatory reporting.

Summary - Sam was a great speaker, and for a change it was refreshing to not have presentation slides backing up what the speaker was saying. His thoughts on forward looking models being true risk management and moving away from risk measurement seem to echo those of Ricardo Rebanato of a few years back at RiskMinds (see post). I think his thoughts on P&L motivation being the only way that risk management advances are correct, although I think there is a lot of risk innovation at the banks but at a trading desk level and not at the firm-wide level which is caught up in regulation - the trading desks know that capital is scarce and are wanting to use it better. I think this siloed risk management flies in the face of much of the firm-wide risk management and indeed firm-wide data management talked about in the industry, and potentially still shows that we have a long way to go in getting innovation and forward looking risk management at a firm level, particularly when it is dominated by regulatory requirements. However, having a truly integrated risk data platform is something of a hobby-horse for me, I think it is the foundation for answering all of the regulatory and risk requirementst to come, whatever their form. Finally, I could not agree more easy analysis for end-users is a vital part of data management for risk, allowing business users to do risk management better. Too many times IT is focussed on systems that require more IT involvement, when the IT investment and focus should be on systems that enable business users (trading, risk, compliance) to do more for themselves. Data management for risk is key area for improvement in the industry, where many risk management sytem vendors assume that the world of data they require is perfect. Ask any risk manager - the world of data is not perfect and manual data validation continues to be a task that takes time away from actually doing risk management.

Posted by Brian Sentance | 14 December 2011 | 11:29 pm

A-Team event – Data Management for Risk, Analytics and Valuations

My colleagues Joanna Tydeman and Matthew Skinner attended the A-Team Group's Data Management for Risk, Analytics and Valuations event today in London. Here are some of Joanna's notes from the day:

Introductory discussion

Andrew Delaney, Amir Halton (Oracle)

Drivers of the data management problem – regulation and performance.

Key challenges that are faced – the complexity of the instruments is growing, managing data across different geographies, increase in M&As because of volatile market, broader distribution of data and analytics required etc. It’s a work in progress but there is appetite for change. A lot of emphasis is now on OTC derivatives (this was echoed at a CityIQ event earlier this month as well).

Having an LEI is becoming standard, but has its problems (e.g. China has already said it wants its own LEI which defeats the object). This was picked up as one of the main topics by a number of people in discussions after the event, seeming to justify some of the journalistic over-exposure to LEI as the "silver bullet" to solve everyone's counterparty risk problems.

Expressed the need for real time data warehousing and integrated analytics (a familiar topic for Xenomorph!) – analytics now need to reflect reality and to be updated as the data is running - coined as ‘analytics at the speed of thought’ by Amir. Hadoop was mentioned quite a lot during the conference, also NoSQL which is unsurprising from Oracle given their recent move into this tech (see post - a very interesting move given Oracle's relational foundations and history)

Impact of regulations on Enterprise Data Management requirements

Virginie O’Shea, Selwyn Blair-Ford (FRS Global), Matthew Cox (BNY Melon), Irving Henry (BBA), Chris Johnson (HSBC SS)

Discussed the new regulations, how there is now a need to change practice as regulators want to see your positions immediately. Pricing accuracy was mentioned as very important so that valuations are accurate.

Again, said how important it is to establish which areas need to be worked on and make the changes. Firms are still working on a micro level, need a macro level. It was discussed that good reasons are required to persuade management to allocate a budget for infrastructure change. This takes preparation and involving the right people.

Items that panellists considered should be on the priority list for next year were:

· Reporting – needs to be reliable and meaningful

· Long term forecasts – organisations should look ahead and anticipate where future problems could crop up.

· Engage more closely with Europe (I guess we all want the sovereign crisis behind us!)

· Commitment of firm to put enough resource into data access and reporting including on an ad hoc basis (the need for ad hoc was mentioned in another session as well).

Technology challenges of building an enterprise management infrastructure

Virginie O’Shea, Colin Gibson (RBS), Sally Hinds (Reuters), Chris Thompson (Mizuho), Victoria Stahley (RBC)

Coverage and reporting were mentioned as the biggest challenges.

Front office used to be more real time, back office used to handle the reference data, now the two must meet. There is a real requirement for consistency, front office and risk need the same data so that they arrive to the same conclusions.

Money needs to be spent in the right way and fims need to build for the future. There is real pressure for cost efficiency and for doing more for less. Discussed that timelines should perhaps be longer so that a good job can be done, but there should be shorter milestones to keep business happy.

Panellists described the next pain points/challenges that firms are likely to face as:

· Consistency of data including transaction data.

· Data coverage.

· Bringing together data silos, knowing where data is from and how to fix it.

· Getting someone to manage the project and uncover problems (which may be a bit scary, but problems are required in order to get funding).

· Don’t underestimate the challenges of using new systems.

Better business agility through data-driven analytics

Stuart Grant, Sybase

Discussed Event Stream Processing, that now analytics need to be carried out whilst data is running, not when it is standing still. This was also mentioned during other sessions, so seems to be a hot topic.

Mentioned that the buy side’s challenge is that their core competency is not IT. Now with cloud computing they are more easily able to outsource. He mentioned that buy side shouldn’t necessarily build in order to come up with a different, original solution.

Data collection, normalisation and orchestration for risk management

Andrew Delaney, Valerie Bannert-Thurner (FTEN), Michael Coleman (Hyper Rig), David Priestley (CubeLogic), Simon Tweddle (Mizuho)

Complexity of the problem is the main hindrance. When problems are small, it is hard for them to get budget so they have to wait for problems to get big – which is obviously not the best place to start from.

There is now a change in behaviour of senior front office management – now they want reports, they want a global view. Front office do in fact care about risk because they don’t want to lose money. Now we need an open dialogue between front office and risk as to what is required.

Integrating data for high compute enterprise analytics

Andrew Delaney, Stuart Grant (Sybase), Paul Johnstone (independent), Colin Rickard (DataFlux)

The need for granularity and transparency are only just being recognised by regulators. The amount of data is an overwhelming problem for regulators, not just financial institutions.

Discussed how OTCs should be treated more like exchange-traded instruments – need to look at them as structured data.

Posted by Brian Sentance | 17 October 2011 | 11:44 pm

Internal model approval, risk management and regulatory compliance

Achieving regulatory approval can be challenging if we consider that regulators are concerned about both the risk calculation methodology in place but also the quality, consistency and auditability of the data feeding the risk systems used for regulatory reporting.

The data management project at LBBW (Landesbank Baden-Württemberg), for example, was initiated to support LBBW’s internal model for market risk calculations, combined with the additional aim of enabling risk, back office and accountancy departments to have transparent access to high quality and consistent data.

This required a consolidated approach to the management of data in order to support future business plans and successful growth and we worked with LBBW to provide a centralised analytics and data management platform which could enhance risk management, deliver validated market data based upon consistent validation processes and ensure regulatory compliance.

More information on the joint project at LBBW can be found in the case study, available on our website. Any questions, drop us a line!

Posted by Sara Verri | 22 September 2011 | 6:21 pm

Data Unification - just when you thought it was safe to go back in the water...

Sitting by the sea, you have just finished your MATLAB reading and now are wondering what to read next?

No worries!

We have just published our "TimeScape Data Unification" white paper. Not a pocket edition I am afraid, but some of you may find it interesting.

It describes how - post-crisis - a key business and technical challenge for many large financial institutions is to knit together their many disparate data sources, databases and systems into one consistent framework than can meet the ongoing demands of the business, its clients and regulators. It then analyses the approaches that financial institutions have adopted to respond to this issue, such as implementing a ETL-type infrastructure or a traditional golden copy data management solution.

Taking on from their effectiveness and constraints, it then shows how companies looking to satisfy the need for business-user access to data across multyple systems should consider a "distributed golden copy" approach. This federated approach deals with disparate and distributed sources of data and should also provide easy and end-user interactivity whilst maintaining data quality and auditability.

The white paper is available here if you want to take a look and if you have any feedback or questions, drop us a line!

Posted by Sara Verri | 27 July 2011 | 3:19 pm

MATLAB - The perfect read for the beach...

For those who are wondering what summer reading to take on holiday, we have just published our white paper "TimeScape and MATLAB", a pocket edition which outlines how TimeScape and MATLAB can be combined to provide enhanced data analysis and visualisation tools to financial organisations.

Whilst swimming in the blue ocean, walking in the countryside or enjoying a new country, take a break and find out how TimeScape's best of breed data capture and storage can be combined with the analytical capabilities of MATLAB to produce compelling solutions to real-world problems encountered within financial services.

Ok, ok, kidding here. Just go on holiday and enjoy your time off from complex financial problems!

But when you are back or if you are very interested (or sadly not going on holiday soon), please take a look at our white paper. It details how:

TimeScape data and analytics can be accessed from MATLAB
MATLAB computational and visualization tools can be used to manipulate and analyse TimeScape data
Complex data sets generated in MATLAB can be saved back to TimeScape for persisted storage
MATLAB components can be called from TimeScape to enrich TimeScape hosted functionality

and much more.

Feel also free to suggest this summer reading to your friends (or enemies!).

Posted by Sara Verri | 22 July 2011 | 2:40 pm

PRMIA on Data and Analytics

Final presentation at the PRMIA event yesterday was by Clifford Rossi and was entitled "The Brave New World of Data & Analytics Following the Crisis: A Risk Manager's Perspective".

Clifford got his presentation going with a humorous and self-depricating start by suggesting that his past employment history could in fact be the missing "leading indicator" for predicting orgnisations in crisis, having worked at CitiGroup, WaMu, Countrywide, Freddie Mac and Fannie Mae. One of the other professors present said that he didn't do the same to academia (University of Maryland beware maybe!).

Clifford said that the crisis had laid bare the inadequacy and underinvestment in data and risk technology in the financial services sector. He suggested that the OFR had the potential to be a game changer in correcting this issue and in helping the role of CRO to gain in stature.

He gave an example of a project at one of the GSEs he had worked at called "Project Enterprise" which was to replace 40 year old mainframe based systems (systems that for instance only had 3 digits to identify a transaction). He said that he noted that this project had recently been killed, having cost around $500M. With history like this, it is not surprising that enterpring risk data warehousing capabilities were viewed as black holes without much payoff prior to the crisis. In fact it was only due to Basel that data management projects in risk received any attention from senior management in his view.

During the recent stress test process (SCAP) the regulators found just how woeful these systems were as the banks struggled to produce the scenario results in a timely manner. Clifford said that many banks struggled to produce a consistent view of risk even for one asset type, and that in many cases, corporate acquisitions had exascerbated this lack of consistency in obtaining accurate, timely exposure data. He said that the mortgage processing fiasco showed the inadequacy of these types of systems (echoing something I heard at another event about mortgage tagging information being completely "free-fromat", without even designated fields for "City" and "State" for instance)

Data integrity was another key issue that Clifford discussed, here talking about the lack of historical performance data leading to myopia in dealing with new products and poor defintions of product leading to risk assessments based on the originator rather than on the characteristics of the product. (side note: I remember prior to the crisis the credit derivatives department at one UK bank requisitioning all new server hardware to price new CDO squared deals given it was supposedly so profitable, it was at that point that maybe I should have known something was brewing...) Clifford also outlined some further data challenges, such as the changing statistical relationship between Debt to Income ratio and mortgage defaults once incomes were self-declared on mortgages.

Moving on to consider analytics and models, Clifford outlined a lot of the concerns covered by the Modeller's Manifesto, such as the lack of qualitative judgement and over-reliance on the quantitative, efficiency and automation superceding risk management, limited capability to stress test on a regular basis, regime change, poor model validation, and cognitive biases reinforced by backward-looking statistical analysis. He made the additional point that in relation to the OFR, they should concentrate on getting good data in place before spending resource on building models.

In terms of focus going forward, Clifford said the liquidity, counterparty and credit risk management were not well understood. Possibly echoing Ricardo Rebonato's ideas, he suggested that leading indicators need to be integrated into risk modelling to provide the early warning systems we need. He advocated that the was more to do on integrating risk views across lines of business, counterparties and between the banking and trading book.

Whilst being a proponent of the OFRs potential to mandate better Analytics and data management, he warned (sensibly in my view) that we should not think that the solution to future crises is simply to set up a massive data collection and Modelling entity (see earlier post on the proposed ECB data utility)

Clifford thinks that Dodd-Frank has the potential to do for the CRO role what Sarbanes-Oxley did in elevating the CFO role. He wants risk managers to take the opportunity presented in this post-crisis period to lead the way in promoting good judgement based on sound management of data and Analytics. He warned that senior management buy-in to risk management was essential and could be forced through by regulatory edict.

This last and closing point is where I think where the role of risk management (as opposed to risk reporting) faces it's biggest challenge, in that how can a risk manager be supported in preventing a senior business manager from seeking a overly risky new business opportunity based on what "might" happen in the future - we human beings don't think about uncertainty very clearly and the lack of a resulting negative outcome will be seen by many to invalidate the concerns put forward before a decision was made. Risk management will become known as the "business prevention" department and not regarded as the key role it should be.

Posted by Brian Sentance | 24 June 2011 | 3:26 pm

IKEA and Market Risk Management – Choice is a worrying thing!

Risk management and data control remain at the top of the agenda at many financial institutions. Many have said that the recent crisis highlighted the need for more consistent, transparent, high quality data management, which I totally agree with (but working for Xenomorph, I would I guess!). Although the crisis started in 2007, it would seem that many organizations still do not have the data management infrastructure in place to achieve better risk management.

I moved apartment last week and had to face the terrifying prospect of visiting IKEA to buy some new furniture. On walking through the endless corridors of furniture ideas I wondered whether the people at major financial institutions feel as I did: I knew I needed two wardrobes, I knew the dimensions of the rooms, I knew how many drawers I wanted. Then I got to the wardrobes showroom, sat in front of the “Create your own wardrobe” IKEA software and the nightmare started. How many solutions are there to solve your problems? And how many solutions, once you get to know of their existence, make you aware of a problem you didn’t know you had? That’s how I spent 2 days at IKEA choosing my furniture and still I wonder whether in the end I got the right solution for my needs.

Coming back to risk management, I imagine the same dilemma may be faced by financial institutions looking to implement a data management solution. How many software providers are out there? What data model do they use? Are they flexible enough to satisfy evolving requirements? How can we achieve an integrated data management approach? Will they support all kind of asset classes, even the most complex?

In these times of new regulations where time goes fast and budget is tight, selection processes have become more scrupulous.

As often happens in life, when we need a plumber for example, or a new dentist, we look for positive recommendations, people willing to endorse the efficiency and reliability of the service. So, with this in mind, please take a look at the case study we put together with Rabobank International, who have been using our TimeScape analytics and data management system at their risk department since 2002 for consolidated data management. More client stories are also available on our website here: www.xenomorph.com/casestudies.

I hope that many of you will benefit from reading the case study and for any questions (on IKEA wardrobes too!), please get in touch...

Posted by Sara Verri | 8 June 2011 | 9:07 am

More formal management of instrument valuation needed

Xenomorph has today released its white paper “Instrument Valuation Management: management of derivative and fixed income valuations in a multi-asset, multi-model, multi-datasource and multi-timeframe environment”.

The white paper expands on the “Rates, Curves and Surfaces – Golden Copy Management of Complex Datasets” white paper Xenomorph published recently (see earlier post) and describes how, despite the increasing importance of instrument valuation to investment, trading and risk management decisions, valuation management is not yet formally and fully addressed within data management strategies and remains a big concern for financial institutions.

Too often, says Xenomorph, valuations (and the analytics used to process input and calculate output data) fall between traditional data management providers and pricing model vendors. This leads to the over–use of tactical desktop spreadsheets where data “escapes” the control of the data management system, leading to an increased operational risk.

Whilst instrument valuation is certainly not the primary cause of the recent financial crisis, the lack of high quality, transparent valuations of many complex securities resulted in market uncertainty and in the failure of many risk models fed by untrustworthy valuations.

“A deeper understanding of financial products reduces operational risk and promotes quality, consistency and auditability, ensuring regulatory compliance”, says Brian Sentance, CEO Xenomorph. “Clients’ requirements have evolved and portfolio managers, traders and risk managers recognize that it is no longer sufficient to treat valuation as an external, black-box process offered by pricing service providers”, he adds.

Nowadays, regulators, auditors, clients and investors demand even more drill-down to the underlying details of an instrument’s valuation. It is therefore important to implement an integrated, consistent analytics and data management strategy which cuts across different departments and glues together reference and market data, pricing and analytics models, for transparent, high quality, independent valuation management.

“Our TimeScape solution provides a valuation environment which offers rapid and timely support for even the most complex instruments, allowing our clients to check easily the external valuation numbers, based on their choice of model and data providers”, says Sentance. “Otherwise, what is the point of good data management if the valuations and the analytics used are not based on the same data management infrastructure principles?”

For those who are interested, the white paper is available here.

Posted by Sara Verri | 4 May 2011 | 12:41 pm

Rates, curves and derived data management remains a neglected area following the crisis

Xenomorph has released its white paper 'Rates, Curves and Surfaces – Golden Copy Management of Complex Datasets'. The white paper describes how, despite the increasing interest in risk management and tighter regulations following the crisis, the management of complex datasets – such as prices, rates, curves and surfaces - remains an underrated issue in the industry. One that can undermine the effectiveness of an enterprise-wide data management strategy.

In the wake of the crisis, siloed data management, poor data quality, lack of audit trail and transparency have become some of the most talked about topics in financial markets. People have started looking at new approaches to tackle the data quality issue that found many companies unprepared after Lehman Brothers' collapse. Regulators – both nationally and internationally – strive hard to dictate parameters and guidelines.

In light of this, there seems to be a general consensus on the need for financial institutions to implement data management projects that are able to integrate both market and reference data. However, whilst having a good data management strategy in place is vital, the industry also needs to recognize the importance of model and derived data management.

Rates, curves and derived data management is too often a neglected function within financial institutions. What is the point of having an excellent data management infrastructure for reference and market data if ultimately instrument valuations and risk reports are run off spreadsheets using ad-hoc sources of data?

In this evolving environment, financial institutions are becoming aware of the implications of a poor risk management strategy but are still finding it difficult to overcome the political resistance across departments to implementing centralised standard datasets for valuations and risk.

The principles of data quality, consistency and auditability found in traditional data management functions need to be applied to the management of model and derived data too. If financial institutions do not address this issue, how will they be able to deal with the ever-increasing requests from regulators, auditors and clients to explain how a value or risk report was arrived at?

For those who are interested, the white paper is available here.

Posted by Sara Verri | 24 February 2011 | 5:45 pm

2010 Risk in Review NY

I went along to a a Prmia event last night "2010 - Risk Year in Review". The event started with a somewhat overwhelming brain dump of economic and credit statistics from John Lonski, Chief Capital Markets Economist at Moody's Analytics. In summary he seems very bullish about corporate credit spreads tightening given the way in which corporate profit growth is surging ahead of debt growth. His main concern for the economy was maybe unsurprisingly the US housing market and whether this will bottom out and start to rise in 2011. Given fiscal imbalances and competition from emerging markets he did not think that inflation was a big risk despite activity such as QE2.

Robert Iommazzo of search firm Seba International did a fairly dry presentation on industry compensation for risk managers. Seba seem to getting around having had a big presence at Riskminds in Geneva last week. This section only livened up when the questions started after the presentation, and is probably worth noting that the UK FSA is being perceived as a "Big Brother" with its involvement in setting compensation policies in financial markets. Obviously the FSA is not heading back to the heady days of the 1970's where central government set industry pay rises (journalists please note this meant you back then!), but it is also obvious that such control over an individual's remuneration is something that goes totally contrary to an American way of thinking. UK Government needs to be mindful of this perception particularly if it leaves itself open to arbitrage on compensation policy from other financial centres.

Panel debate followed, involving Ashish Das of Moody's, Yury Dubrovsky of Lazard Asset Management, Jan H. Voigts of the NY Fed and Christopher Whalen of Institutional Risk Analytics. Main points:

Chris said that he was one who was predicting a further fall in the housing market next year, and he asked the audience that when they looked at economic statistics, credit spreads,the Vix, bond spreads, did anyone getting the feeling the things are "normal" yet? Using these numbers and plugging them into a model does any believe the results are stable and can be relied upon? The audience fundamentally seemed to agree with these "warning" questions.
Jan asked the audience to consider how believable is your data and to try to understand what data is critical for your business and that is imperative to create tools to manage this data appropriately. Jan said that the biggest challenge for financial institutions going forward is how to calibrate what rate/volume/type of business you can transact safely and that this needed a lot more consideration.
Yury said that he finds that the risks present in 2008 are still around in 2010, but now with the addition of European sovereign credit problems and the raft of regulation heading towards the industry. To add to this pessimistic note, he also said that some of the interest in "hot" emerging markets such as the BRICs was resulting in investments in lower quality IPOs relative to previous years.
Ashish thought that systemic risk was going to become more important for the industry. With the setting up of the Office of Financial Research (OFR) next year, he suggested that the industry needed to take much more of a lead in sorting out its own house in advance of letting the regulators do so. On the subject of models, he said that models should supplement human judgement but not replace it, and mentioned the quote by George E. P. Box that "all models are wrong, but some are useful".
Chris suggested that the role of risk managers will become more like that of a credit collector, with more involvement in actually seeing what can be recovered once a default has occurred. He also suggested that the industry should create its own consensus-based ratings (supplemented by the existing CRAs) to get a more reliable view of credit.
Ashish echoed some of the speakers last week at Riskminds in saying that regulatory compliance is not risk management, and that practitioners should do more to guide the regulators.
On the subject of risk culture, Yury asked how many risk managers knew data, quant, markets and how to deal with the egos of traders and senior management. This last point seemed to be conceded by the audience as a major weakness of the risk management profession and goes back to whether a risk manager is willing to put his career on the line to go against accepted business strategy.
Chris added that having worked at several investment banks he had not yet experienced a risk manager attending a senior committee, let alone a risk manager speaking up against a senior trader. He talked of two business models "Paranoid and Nimble" and "Well Documented and Pedantic" with the second one being the only one possible in his view once a business gets to a certain size.
On the subject of Government Sponsored Enterprises (GSEs like Fannie Mae and Freddie Mac) Chris said that the role of these will be up for review by the end of 2011. He thinks that the banks will head back towards actually holding mortgages and loans and the GSEs will become more conduits rather than direct sources of finance. This was news to me, given that so far the GSEs have been notably left out of recent reviews of what went wrong with the recent crisis.

Panel was very good, all speakers very knowledgeable. "Regulation is not risk", "models are not perfect", "risk governance" and "take control of your data" were all themes that echoed last week's RiskMinds event, allbeit with more of an American rather than international viewpoint on the economy, regulation and markets.

Posted by Brian Sentance | 15 December 2010 | 5:16 pm

Risk USA - 15 cents in the dollar isn't good...

I went along to the Risk USA event yesterday and caught a good panel in the afternoon called “Garbage in, garbage out” Servicing the data supply and analytic needs for risk management.

In particular, one of the speakers, Frank R. Brown, described some work he had done as a consultant at one financial institution on tracking and rebalancing an index product. To do this, Frank had to integrate the constituent instrument symbology of the:

Custodian
Index Provider
Real-Time Data Provider
Rebalancing Software
In-house Trading System

On top of this, corporate events might result in changes to symbology that not all providers would be up to date on, with various lags before all had caught up with the corporate action (rebalancing software often late, custodian often not changing symbol at all). He mentioned that he did all of this symbology management manually in Excel.

Of his time, he said he spent:

65% on managing the symbology and dealing with data issues
20% managing the various vendor APIs in Excel to update the data
15% on tracking and rebalancing

To sum up, he said that a productive work level of 15 cents in the dollar wasn't good value for the client and yet the issue continues on and on. I don't think that his example was particularly earth shattering in terms of newness, but it put in a very simple and pragmatic context the importance of doing some of the simple things right and the benefits of a more automated approach to data management, even before you delve into the data quality/validity issues of the market data itself.

Just to end on an entertaining note, then back to the title of the talk on "Garbage-in, garbage-out..." the panel moderator (Domenic Iannaccone of Sybase) put forward a good quote he had heard:

"If everyone used the same garbage at least that would be a step forward!"

Transparency and consistency can take many forms, but I didn't know it needed to apply to incorrect data too!...

Posted by Brian Sentance | 4 November 2010 | 7:15 pm

A French Slant on Valuation

Last Thursday, I went along to an event organized by the Club Finance Innovation on the topic of “Independent valuations for the buy-side: expectations, challenges and solutions”.

The event was held at the Palais Brongniart in Paris, which, for those who don’t know (like me till Thursday), was built in the years 1807-1826 by the architect Brongniart by order of Napoleone Bonaparte, who wanted the building to permanently host the Paris stock exchange.

Speakers at the roundtable were:

Eric Benhamou, CEO Pricing Partners
Francis Cornut, Président DeriveXperts
Jean-Marc Eber, Président LexiFi
Patrick Hénaff, Associated Professor at the University of Bretagne (see model validation paper for additional background)
Claude Martini, CEO Zeliade Systems

The event focussed on the role of the buy-side in financial markets, looking in particular at the concept of independent valuations and how this has taken an important role after the financial downturn. However, all the speakers agreed that remains a large gap between the sell-side and buy-side in terms of competences and expertise in the field of independent valuations. The buy-side lacks the systems for a better understanding of financial products and should align itself to the best practices of the sell-side and bigger hedge funds.

The roundtable was started by Francis Cornut of DeriveXperts, who gave the audience a definition of independent valuation. Whilst valuation could be defined as the “set of data and models used to explain the result of a valuation”, Cornut highlighted how the difficulty is in saying what independent means; there is in fact a general confusion on what this concept represents: internal confusion, for example between the front office and risk control department of an institution, but also external confusion, when valuations are done by third-parties.

Cornut provided three criteria that an independent valuation should respect:

Autonomy, which should be both technical and financial;
Credibility and transparency;
Ethics, i.e.: being able to resist to market/commercial pressure and deliver a valuation which is free from external influences/opinions.

Independent valuations are the way forward for a better understanding of complex, structured financial products. Cornut advocated the need for financial parties (clients, regulators, users and providers) to invest more and understand the importance of independent valuations, which will ultimately improve risk management.

Jean-Marc Eber, President LexiFi, agreed that the ultimate objective of independent valuations is to allow financial institutions to better understand the market. To accomplish this, Eber pointed to the fact that when we speak about services to clients, we should first think of what are their real needs. The bigger umbrella of “buy-side” implies in fact different needs and there is often a contradiction on what regulators want: on one side, having independent valuations provided by independent third parties; on the other side, independent valuations really mean that internal users/staff do understand what there is underline the products that a company have.In the same way, we don’t just need to value products but also measure their risk and periodically re-value them.It is important, in fact, to have the whole picture of the product being evaluated in order to make the buy-side more competitive.

Another point on which the speakers agreed is traceability: as Eber said, financial products don’t exist just as they are, but they go under transformation and change several times. Therefore, the market needs to follow the products across its life cycle till its maturity stage and this pose a technology challenge, in providing scenario analysis for compliance and keeping track of the audit trail.

At the question, ‘what has the crisis changed’ panellists answered:

Eber: the crisis showed the need to be more competent and technical to avoid risk. He highlighted the need to understand the product and its underlying. Many speak of having a central repository for OTCs, obligations, etc but this needs more thinking from the regulators and the financial markets. Moreover, the markets should focus more on quality data and transparency.

Eric Benhamou, CEO pricing Partners, sees an evolution of the market as the crisis showed underestimated risks which are now being taken in consideration.

Claude Martini, CEO Zeliade, advocated the need for financial markets to implement best practices for product valuations: buy-side should apply the same practices already adopted by the sell-side and verify the hypotheses, price and risk related to a financial product.

Cornut admitted things have changed since 2005, when they launched DerivExperts and nobody seemed to be interested in independent valuations. People would ask what value they would get from an investment in independent valuations: yes, regulators are happy but what’s the benefit for me?

This is changing now that financial institutions know that a deeper understanding of financial products increases their ability to push the products to their clients. The speech I enjoyed the most was from Patrick Hénaff, associated professor at the University of Bretagne and formerly Global Head of Quantitative Analysis - Commodites at Merrill Lynch / Bank of America.

He took a more academic approach and contested the fact that having two prices to confront is thought to reduce the incertitude on the product but highlighting as this is not always the case. I found interesting his idea of giving a product price with a confidence interval or a ‘toxic index’ which would represent the incertitude about the product and reproduce the model risk which may originate from it.

We speak too often about the risk associated to complex products but Hénaff, explained how the risk exists even on simpler products, for example the calculation of VAR on a given stock positioning. A stock is extremely volatile and we can’t know its trend; providing a confidence interval is therefore crucial. What is new instead, it is the interest that many are showing in assigning a price to a determinate risk, whilst before model risk was considered a mere operational risk coming out from the calculation process. Today, a good valuation of the risk associated to a product can result in less regulatory capital used to cover the risk and as such it is gaining much more interest from the market.

Henaff describes two approaches currently taken from academic research on valuations:

1) Adoption of statistic simulation in order to identify the risk deriving from an incorrect calibration of the model. This consists in taking historical data and test the model, through simulations and scenarios, in order to measure the risk associated in choosing a model instead of another;)

2) Have more quality data. Lack of quality data implies that models chosen are inaccurate as it is difficult to identify exactly what model we should be using to price a product.

Model risk, which as said above was before considered an operational risk, now becomes of extremely importance as it can free up capital. Hénaff suggested that is key to find for model risk the equivalent of the VAR for market risk, a normalized measure. He also spoke about the concept of a “Model validation protocol”, giving the example of what happens in the pharmaceutical and biologic sectors: before launching a new pill into the market, this is tested several times.

Whilst in finance products are just given with their final valuation, the pharmaceutical sector provides a “protocol” which describes the calculations, analysis and processes used in order to get to the final value and their systems are organized to provide a report which would show all the deeper detail. To reduce risk, valuations should be a pre-trade process and not a post-trade.

This week, the A-Team group published a valuations benchmarking study which shows how buy-side institutions are turning more and more often to third-parties valuations, driven mainly by risk management, regulations and client needs. Many of the institutions interviewed also admitted that they will increase their spending in technology to automate and improve the pricing process, as well as the data source integration and the workflow.

This is in line on what has been said at the event I attended and confirmed by the technology representatives speaking at the roundtable.

I would like to end with what Hénaff said: there can’t be a truly independent valuation without transparency of the protocols used to get to that value.

Well, Rome wasn’t built in a day (and as it is my city we’re speaking about, I can say there is still much to build, but let’s not get into this!) but there is a great debate going on, meaning that financial institutions are aware of the necessity to take a step forward. Much is being said about the need for more transparency and a better understanding of complex, structured financial products and still there is a lot to debate. Easier said than done I guess but, as Napoleon would say, victory belongs to the most persevering!

Posted by Sara Verri | 28 October 2010 | 4:50 pm

Analytics Management by Sybase and Platform

I went along to a good event at Sybase New York this morning, put on by Sybase and Platform Computing (the grid/cluster/HPC people, see an old article for some background). As much as some of Sybase's ideas in this space are competitive to Xenomorph's, some are very complimentary and I like their overall technical and marketing direction in focussing on the issue of managing of data and analytics within financial markets (given that direction I would, wouldn't I?...). Specifically, I think their marketing pitch based on moving away from batch to intraday risk management is a good one, but one that many financial institutions are unfortunately (?) a long way away from.

The event started with a decent breakfast, a wonderful sunny window view of Manhattan and then proceeded with the expected corporate marketing pitch for Sybase and Platform - this was ok but to be critical (even of some of my own speeches) there is only so much you can say about the financial crisis. The presenters described two reference architectures that combined Platform's grid computing technology with Sybase RAP and the Aleri CEP Engine, and from these two architectures they outlined four usage cases.

The first use case was for strategy back testing. The architecture for this looked fine but some questions were raised from the audience about the need for distributed data cacheing within the proposed architecture to ensure that data did not become the bottleneck. One of the presenters said that distributed cacheing was one option, although data cacheing (involving "binning" of data) can limit the computational flexibility of a grid solution. The audience member also added that when market data changes, this can cause temporary but significant issues of cache consistency across a grid as the change cascades from one node to another.

Apparently a cache could be implemented in the Aleri CEP engine on each grid node, or the Platform guy said that it was also possible to hook in a client's own C/C++ solution into Platform to achieve this, and that their "Data Affinity" offering was designed to assist with this type of issue. In summary their presentation would have looked better with the distributed cacheing illustrated in my view, and it begged the question as to why they did not have an offering or partner in this technical space. To be fair, when asked whether the architecture had any performance issues in this way, they said for the usage case they had then no it didn't - so on that simple and fundamental aspect they were covered.

They had three usage cases for the second architecture, one was intraday market risk, one was counterparty risk exposure and one was intraday option pricing. On the option pricing case, there was some debated about whether the architecture could "share" real-time objects such as zero curves, volatility surfaces etc. Apparently this is possible, but again would have benefitted by being illustrated first as an explicit part of the architecture.

There was one question about the usage of the architecture applied to transactional problems, and as usual for an event full of database specialists there was some confusion as to whether we were talking about database "transactions" or financial transactions. I think it was the latter, but this wasn't answered too clearly but neither was the question asked clearly I guess - maybe they could have explained the counterparty exposure usage case a bit more to see if this met some of the audience member's needs.

The latter question on transactions above got a conversation going on about resilliancy within the architecture, given that the Sybase ASE database engine is held in-memory for real-time updates whilst the historic data resides on shared disk in Sybase IQ, their column-based database offering. Again full resilience is possible across the whole architecture (Sybase ASE, IQ, Aleri and the Symphony Grid from Platform) but this was not illustrated this time round.

Overall good event with some decent questions and interaction.

Posted by Brian Sentance | 20 October 2010 | 7:40 pm

Dodd Frank Regulation - being seen to be doing something?

I went along to a Six Telekurs event "Securities Valuations: Is the Price Right?" last week - good event with some interesting speakers, most notably Paul Atkins of Patomak Partners to talk about the Dodd-Frank Wall Street Reform and Consumer Protection Act 2010. Paul is based out of Washington and was not very complimentary about what has been going on.

He started by saying that the Act was very large in size, with over 2319 pages (compared to SarbOx with only 60) and given this size he suggested that you could guess how many in Congress had actually read it. Background to the Act were:

"Political Tailwinds" such as:
- New Democrat Government with tenuous majority
- Ambitious legislative plans
- Bleak economic back-drop
An angry populace:
- TARP bailouts/Wall St bonuses
- Recession and high unemployment
- Perception that Govt. contributed to crisis
Aggressive case for new regulation based on:
- Lack of confidence in current systems and regulation
- "Too big to fail" demonstrating that regulators lack the toolsets necessary to deal with such events
- High leverage across the financial system and the economy
- Poor risk management by existing participants
- Opaque shadow banking system and opaque derivatives markets

He summarised that Housing and the Credit Rating Agencies were the key fundamentals behind the financial crisis.

Paul said that with the new regulation had the following features:

The Act is a sweeping revision of financial regulation in the US
- few dodged the regulatory changes (notably insurance managed to do this)
The Federal Reserve has emerged pre-eminent amongst all regulatory bodies in the US.
Significant discretion has been yielded to regulators to work out specifics
Sheer size and ambiguous wording of the Act exacerbates the uncertainty in the market and economy and will require further fixes over coming years
The Act does not reform Government Sponsored Enterprises (Fannie Mae, Freddie Mac)
Far from reducing/simplifying the number of agencies involved in regulation the Act eliminated 1 agency and created 13 more
Paul asked the question whether spreads and volatility will rise in the market due to new regulation (such as the Volcker rule) and whether ultimately this will trickle down to hinder or benefit SMEs.
The Act will likely result in regulatory arbitrage opportunities and Paul said this was not a good thing for the United States

Paul said that in his view Congress learned the wrong lessons from the crisis:

No reform of Fannie Mae and Freddie Mac
Government Housing Policy left unaddressed
Transparency still lacking despite efforts from FASB on fair value
International Policy Co-ordination is still an open question as to its extent
No reform of existing regulator structures
The crisis has resulted in payoffs to favoured groups (Unions, Trial Lawyers etc)

Paul talked about how hedge funds and private equity funds were going to experienced increased regulation with them having to register if they have over $100M assets under management and future implications for systemic risk provisions. He mentioned that Venture Capital investments had escaped being required to register if the lock-up period was over 2 years.

He briefly discussed the coming changes in OTC derivatives on centralised clearing, post trade reporting and new liability provisions. Paul was also concerned about certain SEC related issues such as "Whistleblower" provisions which contain a bounty programme of about 10-30% of any fine subsequently awarded against a financial institution. He re-iterated that it was not yet clear what all of the bodies involved in regulation would be doing, and at the same time as this was the case the very same bodies were also being given very strong powers such as that of legal subpoena.

Paul was a very knowledgeable speaker and had some good points to make. Listening to him speak it would seem from my perspective that the Act is a prime example of "being seen to be doing something" to address the crisis rather than something better structured, with all of "law of unintended consequencies" risks that such an initiative entails.

Posted by Brian Sentance | 14 October 2010 | 7:32 pm

The Humans Between Risk and Data

Some of my thoughts on risk management, data management and human behaviour, are to be found on page 20 of the Inside Reference Data Special Report "Managing Risk"

Posted by Brian Sentance | 21 June 2010 | 1:22 pm

A Crisis Needs a Utility?

I heard Francis Gross of the ECB speak at one of the panel events at the XTrakter Conference last week, and found that I couldn't avoid asking him whether the aims of the "Data Utility" initiative by the ECB could be better separated from the means by which the ECB proposes to solve them. At the moment, reference data issues for the industry and the data utility seem to be presented as a single "package". I can't say that the response to my question was a clear one to my understanding; however I would say that Francis was helpful after the panel had finished and provided a recent presentation of their ideas, of which you can find a copy here.

Looking through the presentation, the motivations put forward for why the industry needs a data utility seem to include:

Data processing must be done in an automated manner, since data volumes have moved beyond the capabilities of manual processing.
- can't see anyone arguing with this
Data is a major bottleneck, with multiple providers/sources each with the own "data dialect"
- agreed and to some extent what keeps data/data management vendors in business, but sounds sensible to standardise if possible as there are plenty of other problems to address
These data dialects lead to increased cost, operational risk and reduced responsiveness
- agreed, mainly a cost aspect I would suggest
The recent crisis was not helped by weak data management in the industry
- but nor was it the cause, so not a great premise for a data utility
- lack of transparency of data
  - "transparency" is an over-used word at the moment, but certainly clarity and quality were/are needed
- systematic risk could not be assessed due to the availability of data
  - using terms like "systematic risk" seems to imply the regulators could calculate something, whereas this discipline is new so I guess we are really talking about simply knowing who is exposed to who and how.
We need the capability to run large scale computing analysis on a vast pool of micro data, sometimes on an ad-hoc basis when a crisis begins
- fundamentally agreed but also good to qualify with what you propose to be calculated - having a set of "numbers" doesn't seem to have helped much recently...

I started the above bullet point list by saying it contains the motivations for "why the industry needs a data utility" but I guess looking at the above list they really point to the more general aim of "why we need better industry-level data management". In the presentation the above points are then used to state:

"We all need the same good basic reference data. Why build more than one infrastructure?"

Maybe "Why build more than one infrastructure?" should really be changed to say "Why maintain more than one infrastructure?" given that Bloomberg, Thomson Reuters, Six Telekurs, Interactive, Markit and all the other vendors already infrastructure to do this. Not sure if I should read anything into the wording but more logical leaps of faith are to follow.

The presentation then moves on to state that shared reference data standards are a must, to which I cannot see many consumers of data disagreeing with that statement. Not sure I agree though with the overly simplistic statement that "Data will be good for all users or good for none". Trying telling that to the accountancy and risk departments for example but I suppose what we are talking about here is basic reference data not the more subjective price and valuation data. Reference data on instruments and entities is either right or wrong, and the presentation makes the good point that no amount of "data cleaning" can help this i.e. if wrong, the data needs to be re-captured from an accurate source.

The call for the establishment and use of reference data standards in the presentation then seems to be used to "slide "into a call for a standard reference data infrastructure. Unless I am very much mistaken, these two things are not necessarily the same thing and so it seems a logical leap has been taken here. The presentation talks about the possible necessity of "top down" legal compulsion for the industry, again something that I could agree and see the need for, but both the issues and legal compulsion do not automatically drive us to a "data utility" as the only option? Why couldn't legal compulsion be applied to the existing data vendors to standardise on common IDs for instance? ISIN is proposed as a standard in the presentation, but I can only assume that this is due to the ECB being mainly focussed on the bond world where to a large degree ISIN's work (i.e. are unique), whereas in the world of equities ISIN needs a lot of qualification (currency, exchange, share class...) before it uniquely identifies a quoted equity.

In summary, the presentation starts with showing how great the ECB's Centralised Security DataBase is (7 million securities, 3 million record updates/day etc...) and it does look good. The data issues for the industry seem clear, although I think the "crisis" is a bit of a red herring to the aim of data cost reduction, however the logical jump from industry need to effectively "we must have a data utility" is an interesting one, one where I would prefer that more options were discussed. It seems ironic that in these days of "transparency" it is not at all that transparent to me why more alternative solutions are not being discussed and a choice justified. Talking of choice and as a final thought, I am also not sure why the data vendors are not up in arms about this initiative - are they frantically lobbying behind the scenes? - do they simply think the utility won't go ahead? - or are they afraid of upsetting the EU? Any insight is very welcome, and maybe more of update from me when I get chance to speak with Francis in more detail.

Posted by Brian Sentance | 4 June 2010 | 7:00 am

XTrakter Conference

I went along to the XTrakter Annual User Conference in London on Thursday - Good event with some great speakers. Angela Knight, CEO of the British Bankers Association, gave a talk to start off the day. Angela seemed a lot less on the defensive than when I have heard her on national radio here in the UK, usually being interrogated by some journalist who wants answers to difficult questions on the financial crisis and the banks role within it.

Angela said that we were in year 3 of the "crisis" with 2008 being about the banks, 2009 being about governments and politics and 2010 being the year of sovereign debt. I guess she enjoyed saying this but that everyone is blaming "Anglo Saxon Banking" for our problems and yet it was not the banks that contributed to the fundamental problems that Greece is facing.

One major theme of her talk was decidedly Euro-Sceptic in tone, which was that the UK idea of internationality and international trade was different from that of Europe. She perceived that in the UK one of our trading parties is Europe, whereas international trade in Europe was more about inter-European and not world-wide trade - I think that there are elements of truth in this but not sure that Germany industry for example would agree that it is not conscious of truly "global" trade? She said that she was concerned by the rules and regulation being put up by governments, particularly in respect of there being too much and in too short a time.

Angela was an engaging speaker and at the very least her opinions prompt reaction, however I have to end this quick post with the best quote of the morning from Anthony Belchambers, CEO of the Futures and Options Association. Anthony said that current frenzy around political and regulatory initiatives to control the financial markets remind him of:

"A bar room brawl, where the brawlers don't punch the person that started the fight, they punch the person they have always wanted to punch..."

Posted by Brian Sentance | 24 May 2010 | 3:11 pm

Counterparty Event

I went along to a morning panel on counterparty data management on Tuesday, sponsored by GoldenSource, Avox and Interactive Data, and hosted by Virginie O'Shea of the A-Team. Counterparty data obviously has a very high profile currently in light of recent events, however the advice from the panel fundamentally seemed to be get the basics of data management right (ownership, control, consistency, quality, transparency), rather than anything radically new.

There was some debate about the possible extension of BIC (Bank Identifier Code) to be used more generally as a standard for a unique business entity identifier - this seemed to be received well but there were concerns that such an initiative would not solve the problem but rather become an addition to the already complex entity-mapping process.

The "Data Utility" from the ECB was also debated, and it was refreshing to here some negative (realistic?) things said about it, such as the concern raised by Interactive that this might involve huge public spend without necessarily understanding why a new government sponsored entity would be able to do better than existing data providers. Obviously a data provider would say that, but I have to agree, it seems there is too much focus on having a data utility and not looking at the different options for solving industry data issues (one option obviously being a data utility, but lets not pre-package the problem with a solution but more of that in later posts...).

For more detail on the event, then take a look at Virginie's blog post.

Posted by Brian Sentance | 21 May 2010 | 9:56 am

Cloudy definitions

Given that I am English and can tend to start many personal introductions with a short conversation about the weather (generally either "awful" or "not bad for this time of year"...), then maybe I should be very receptive to the use of weather-related expressions in technology such as the "cloud". Maybe not however since the "cloud" and "cloud computing" have reached that zenith of marketing hype, when everyone is talking about a new technology regardless of if they are sure what it actually is (or might be, or could become...).

Anyway, I finally swallowed my cynicism and on Thursday morning went along to "Migrating Business to the Cloud", an event by Microsoft hosted at Bafta (small venue where the UK deals out its equivalent (?) of the Oscars). The master of ceremonies was Mark Taylor of Microsoft, who gave a general introduction to what Microsoft are doing in the "cloud", and of particular note he described the four types of computing scenarios where cloud computing can optimally be applied:

Predictable Bursting - where computing needs come and go in predictable waves of usage/demand
Growing Fast - where computing needs are rising exponentially like in a successful internet start-up
Unpredictable Bursting - where computing demand comes in unpredictable bursts, such as that associated with say usage of a backup computer centre in disaster recovery
On and Off - where you might run a process once a month or at an interval you decide

The above definitions seem ok to me but there is (probably understandably) some overlap in usage cases. The "Growing Fast" case for start-ups is interesting and more of that later.

Mark handed over to David Chappell who gave his perspective on cloud platforms as they are today in the market. David was a very entertaining and knowledgeable speaker, despite wearing a dodgy suit (what happened to those trousers?!) and having a peculiar wide foot stance when speaking. Anyway I digress, on to what he said. David started by saying what the "Cloud" is comprised of:

Cloud Applications - basically this is Software as a Service (SaaS) and some current examples of this would be Salesforce.com CRM, Microsoft Exchange Online and Google Apps.
Cloud Platforms - a platform for developing cloud applications, with the following characteristics that it:
- is aimed at developers for creating and running cloud applications, not end consumers
- provides self-service access to computing resources
- allows very granular, on-demand allocation of computing resources
- charges for the consumption of computing resources in a very granular manner

David then explained that due to its ambiguity he disliked the usage of the term "Private Cloud" in the ongoing debate about publicly available cloud services (such as those provided my Amazon, Microsoft and Google) vs. private clouds deployed within private institutions. David said the main difference was that private clouds do not have the economics of public clouds (i.e. pay for what you use only when you need it). That point seemed straightforward, however I would have thought that with a large global organisation with many different departmental computing demands the economics of a private cloud would be similar to a public one.

David then went on to explain that there are two kinds of Cloud Platform:

Infrastructure as a Service (IaaS) - this is a cloud platform the provides a developer with a virtual machine (VM) that has (almost) full access within it; put another way the development environment gives the developer total control but with that control comes responsibility.
Platform as a Service (PaaS) - this is a cloud platform that runs an application that a developer has created; it is easy to use but has limited control for the developer.

David put forward that there has been only 5 major software technology platforms over the past 50 years:

Mainframe
Mini-Computer
PC
PC-based Server
Mobile

He perceives that the Cloud is the 6th major software technology platform, and as such he is extremely enthusiastic about the opportunity and benefits that this presents to the whole of the software industry and its consumers.

David categorised Microsoft's cloud platform as (mostly) PaaS, which had three main components:

Windows Azure - for environment for running cloud applications within the platform
SQL Azure - relational storage within the platform
Windows Azure Platform AppFabric – (David noted the long name and sympathised with trying to name things sensibly) this provides and manages the infrastructure within the platform

He then moved on to describe the main usage scenarios for Windows Azure, for applications that:

need massive scale, such as Web 2.0 applications
need high reliability
have highly variable loading
have short or unpredictable lifetimes
need parallell processing
will either fail fast or scale fast
do not fit easily in a single organisation's data centre, such as joint venture
need external storage

David said that in the fail quickly or scale quickly scenario, this was squarely aimed at technology start-ups where using Cloud technologies would effectively increase the frequency at which new ideas could be tried out at less economic cost if they go wrong, but are ready to scale massively if they become the new "Facebook" - so much so that many of the VCs in Silicon Valley are now insisting that start-ups use cloud technology as a condition of funding.

Amazon's Elastic Compute Cloud (Amazon EC2) was the first major commercial cloud platform, and David categorised this as IaaS, where effectively you get a Virtual Machine (VM) environment that provides a lot of control but requires more effort to control than an PaaS such as Azure.

David said that he was surprised that the Google App Engine, which has Python and now Java as its programming languages, did not come with any traditional relational storage (unlike most other cloud platforms) but on speaking with Google he found that the storage engine and the whole platform is again designed primarily for Web 2.0 apps and as such storage usage was more about retrieving photos, video etc and less about querying across many records.

David was very complimentary about the cloud platform from Salesforce.com called Force.com, He said that the sales pitch from Salesforce.com would be straight to business users, effectively saying that they could build scaleable, resilient applications without involving the IT department and without needing programming expertise. He asked the audience if anyone had used these tools and a few folks confirmed that they were extremely impressed by what the platform offered.

Bob Muglia (President, Server and Business Tools, Microsoft) then gave a quick talk on Microsoft's plans for Azure. He mentioned how Microsoft's new search engine, Bing, was based on several hundred thousand servers running in Azure, but only had a handful of operating staff in contrast with the usual economics (taken from Gartner) that usually 1 operations person was needed for every 50 servers. He emphasised that Microsoft was committed to the further development of "on premises" operating systems but that Microsoft was totally committed to cloud computing, its development and its support.

He said that some of the tools found in the Microsoft technology suite, such as SQL Reporting Services, are not yet available in the cloud on Azure/SQL Azure (due end of year though) - he said that he hoped that people understood that re-engineering an existing application for the cloud sometimes took time to ensure the scaleable and reliability demanded when providing the functionality through the cloud. The vision put forward by Bob for development of cloud applications seemed very compelling, with Microsoft aiming to make things such enabling resilience for a globally available cloud application as simple as ticking a check-box in Microsoft Visual Studio. He put forward that the major barrier to cloud adoption was the human aspect of trust of moving applications "off premises". He said that he saw a fundamental shift across all industries to cloud development and deployment, but added there may be some areas such as government and finance where this process takes a lot longer.

The event then switched to presentations by EasyJet, RiskMetrics and SeeTheDifference. The head of IT at EasyJet gave his pitch first. His department get an annual budget of 0.75% (small?) of turnover of £2.5bn (larger, so translating to £18.75m) and has around 60 people. He presented how EasyJet has taken an incremental approach to the adoption of cloud computing, utilising both "on-premises" and cloud ("off-premises") technology together (exposing end points of applications into the cloud at first). He advised this approach since it:

was a smaller step than full-blown adoption
was lower risk
demonstrated big value in a short time-frame
leveraged the rich functionality available in Azure
accelerated acceptance of cloud technology

Dr Rob Fraser of RiskMetrics was next up. He explained whilst Moore's Law says that computing power doubles every 18 months, the calculations needed for risk management have doubled every six months. This has driven the need for parallel computing to meet this calculation need, and that RiskMetrics' RiskBurst service uses around 2,500 64-bit Opteron cores in their data centre but combines this with use of Azure to meet the peaks in calculation needed during each day (the similarities with power consumption management were pretty apparent). He said that average CPU consumption was around 18% of peak, hence a combination of both on and off premises compute power was a good solution for them. He mentioned that the management of this hybrid combination of technologies, and in particular being able to show real-time billing for it was a key area of investment for RiskMetrics.

The final presentation was by SeeTheDifference. The main point of this presentation was that this charitable organisation had zero permanent staff involved in IT, but regardless was able to deliver a very professional, reliable and scaleable website using external consultants to build on Azure.

Final section of the morning was a roundtable discussion with questions from the audience. The EasyJet guy said that the human mindset was key to the adoption of cloud computing. In terms of what keeps him awake at night was the thought that what would happen/how would attitudes change if any of the cloud infrastructure failed - so far it has experienced 100% up time. Rob of RiskMetrics was concerned about the stability of the platform, trying to ensuring that any changes introduced do not damage reliability. He added that he disagreed with Bob Muglia and thought that financial institutions would adopt public clouds quickly – he cited their experience of their revenues now being 90% based from service provision not on-premises applications. David said that he took some of the comments from Bob to indicate that Microsoft would also offer more of a pure VM (IaaS) soon in addition to the PaaS approach of Azure. David said that trust was the major issue in cloud adoption and he advised an incremental approach so "get your feet wet" then build from there.

On the whole the presentations were good and my knowledge of cloud technology has improved a bit - certainly it is fantastically appealing to develop globally available applications with no scaling, no resilience or data replication issues - it sounds too good to be true which generally means it is, so I guess there is much more work to do in gaining trust and acceptance for this technology. So my (pragmatic?) cynicism remains - but cloudy days are certainly coming and for a change maybe this is something to very much look forward to.

Posted by Brian Sentance | 17 May 2010 | 8:37 am

CEP - Part of the technology furniture?

The CEP market is apparently maturing - don't miss this post "CEP: LaserDisc or DVD?" by Adam Honoré at Aite Group with an interesting view of the future of CEP technology.

Posted by Sara Verri | 29 March 2010 | 11:29 am

Data models are not what they used to be...

AIM have released the results from their 2009 survey on reference data management which is worth a look, particularly given the 2008 results are also shown for comparison. Seems like Mike Atkin and the EDM Council have their work cut out in getting the Semantics Repository adopted if the survey is anything to go by, with the number of institutions using standards-based data models having dropped significantly when comparing 2009 to 2008. What is going on there in these heady days of the finance industry sorting out its data problem through adopting standards? - In cash starved times, maybe it costs more to conform to a standard? - Is the survey data not broad enough? Any ideas appreciated!

Posted by Brian Sentance | 18 March 2010 | 8:09 pm

Risk, Data Transparency and the MBS Market

I spent the morning yesterday over at the FIMA USA event in New York, and caught the panel discussion chaired by Neil Edelstein of GoldenSource. Stand out speakers were Amy Hawkins of BNY Mellon and John Bottega of the Federal Reserve.

Neil started the panel by asking the panel for their thoughts on the current drive to improve "data management for risk". Transparency and quality were mentioned a lot unsurprisingly, with John Bottega adding that he was aware that a lot of banks were now focussed on the data that in the past had been "not available" for risk management, not just the quality of data that is readily accessible. All panelists focussed on the need to manage risk across the whole institution, not just by product silo.

On the topic of data standards and transparency, John referred the audience to testimony on the Mortgage Backed Securities (MBS) market presented to the US Government by the XBRL group. Apparently the filing process for mortgages allows free format filing and so is of little use from an automated processing point of view. John also pointed out that a key piece of data in assessing risk is that the "first time buyer" flag was found to be present in only 15% of the filings.

John also mentioned that if loans and mortgages could be given standard identifiers, then this would enable new levels of risk management - for instance it should be able to extract those obligations against a specific region that for example is experiencing economic recession. These would be the benefits of getting data standards in place.

As was later expanded upon in a later talk by Kay Vicino of Northern Trust, there was a lot of panel discussion on organisational data governance and the management structures needed to achieve it. On the governance side of things then whilst it is not an exciting topic, it is obviously vital - main point seems to be establishing data ownership and responsibilities which brings me back to the point that a lot of (most?) data management issues are down to managing people and organisational politics, not just down to good technology (although it helps!).

Overall a reasonable panel, and the XBRL testimony looks worth a more detailed read (if the testimony link doesn't work then go to the www.xbrl.org site and search for a report called "Using Standards for Transparency")

Posted by Brian Sentance | 17 March 2010 | 4:42 pm

Data Management Panel

Thomson Reuters held a panel event on data management at their London offices on Tuesday last week, with speakers from Barcap, LCH.Clearnet, DB, Mizuho and Citi. This event was held in follow up to their recent report "Beyond Golden Copy". Below are some of my notes on the summary points the panelists made:

The Value of Data - Kris Bhattacharjee of Barcap said that there were currently two main drivers behind the perceived business value of data; i) Regulators are expecting more information, adding additional requirements and conducting more adhoc reporting requests. ii) Business users/decision makers want more granular understanding of trading and risk management data, in order to decide how best to allocate scarce capital to what trading positions.
Data Metrics - Kris said that the metrics were many but timeliness of data was becoming a key metric - over the past two years regulators have moved from allowing say 2 months as a reporting timeline down to 10 days recently. Additionally timeliness is again vital as regulators demand adhoc reporting in response to market events.
Accuracy/Completeness - Again regulators are driving this, with the "bad numbers in, bad numbers out" as the main motivation. Unsurprisingly, counterparty data is also being required at a new level of detail and accuracy down to a portfolio level in light of the crisis.
Granularity of Data - Deeper granularity of data being driven by scarce capital and the need to understand how efficiently it is being used. Basel II has also driven greater granularity over Basel I. Reflecting what I have heard from some our clients, Kris added that the data associated with securitised products had increased greatly as people need to understand exposure/risk and pricing in more detail (rather than assume blanket statistical behaviour for a whole basket of assets).
Stress Scenarios - Kris again mentioned the understanding of counterparty exposure driving the need for new data sets, as had the initiative of banks having "living wills" to allow a bank to be wound down in an orderly manner.
Everybody has Left the Building! - Martin Taylor of LCH.Clearnet was a great speaker and said that the biggest new problem that the collapse of Lehman's created was that ordinarily there are people around to help with extracting from systems what the exposure is to the various counterparties. In the Lehman's case there was nobody around to help, making the process very difficult and leading to the need for changes to address this problem.
Mandating Data Integrity - Martin added that data security, integrity and auditabiliy were vital, and in particular put emphasis on the people that are running the systems that they have their own form of integrity so that an institution knows that the people can trusted but is also capable to deal with a situation where the people are not around to help. Martin felt that this level of data management should be mandated on the industry and that there was an awful lot that finance could learn from industries such as Pharmaceuticals in terms of product approval and management/robustness of data.
Data with No Cost or Value - Neil Fletcher of DB was another good speaker who started his talk by saying that pre-crisis people thought of data as project based, otherwise dealt with it on an adhoc basis and considered data as having no cost or value. Institutions had a spaghetti approach to data, with systems/projects being process not data based i.e. the systems get only the isolated data sets they need only when they need it.
Quality is Now the Data Driver - Neil said that 18 months on from the crisis, then whilst ROI is still important for data projects then quality of data is the key driver.
Sponsorship and Ownership of Data - Neil added that quality data is an asset as are the systems that produce data quality, and to ensure success data management projects needed high level business sponsorship, but also ongoing and clearly defined ownership of all data sets and their quality.
Enterprise Data Virtualisation - Neil said that DB were embarking on a long term project to ensure that all systems get data from the same logical place on a global basis, and that they were investing heavily in data virtualisation technology as a key means of achieving this goal. DB are starting with reference data, moving to transactional/positional data and on to other data types. For each type/category of data ownership would be clearly defined across all systems and would enable real-time transformation of the data into whatever format it is needed in.
Enterprise Data Model - Neil said that as a result of this virtualisation approach then you have to invest in putting together an enterprise data model for all data used in an institution. From my point of view this could be interpreted as a move back to "big EDM" (with all the project risk that implies) but I guess it is being approach on a more staged manner.
Lip Service to Data has Ended - Neil summarised by saying that lip service to data management has ended with the start of the crisis and that 18 months on the enthusiasm for dealing with the data problem has not diminished.
Publish/Validate/Subscribe - Simon Tweddle of Mizuho echoed a lot of what Neil said in approach to global data management and ownership, but added that he believed that the model of publish/subscribe needs to change to publish/validate/subscribe to ensure data quality.

Most of the panelists agreed that bringing in experience from external industries (Pharma, Oil & Gas, Internet Search etc) would be beneficial since we should not assume that the financial market has the expertise to get data management right first time (take a look at this article from the FT for a related idea). Martin of LCH.Clearnet was convinced that mandated data management would come and would be beneficial, which some of other panelists did not agree with and suggested that the industry needs to get ahead of the regulators to head this possibility off. Simon said that the focus on complex data/products was wrong given that the basics (what is our exposure to this counterparty?) were not being done (not sure I agree with this totally, both are needed given the losses from CDOs etc). Overall it was good panel with some interesting debate and speakers.

Posted by Brian Sentance | 8 March 2010 | 2:30 pm

Beyond Golden Copy?

Interesting reading in a survey put together by Lepus and Thomson Reuters and publicised on Finextra this week. Summary findings:

Data management budgets are increasing, with 77% of firms intending to increase spend on data quality and consistency and 32% saying spend would increase significantly.
Tearing down data silos is a key initiative, 70% of firms are looking to revise data management solutions as a result of the crisis, and 31% of firms cited data quality and consistency as the most important driver.
Data management for risk is the top concern, with 87.25% of firms looking to integrate data repositories in risk, and 62.5% saying that they were close/very close.

This seems to be consistent with another article on Finextra this week, with Deloitte predicting a much greater spend on risk management projects. Putting the marketing aspects aside for a moment, I don't think it is abundantly clear from the actual content of the Lepus survey as to why the title includes the phrase "...Beyond Golden Copy" other than the type of data management they refer to seems to have more emphasis on global/firm-wide data integration than your traditional EDM golden copy data warehouse approach.

It is also interesting to hear so much about consistent data across the entire enterprise (driven by risk and regulation) which seems to echo the "big EDM" projects of old that did prove that successful, and to some degree is at odds with what the likes of Golden Source and Asset Control are currently saying about choosing smaller projects to bite off on rather than the enterprise approach. I would suggest however that there is no issue in having smaller projects in mind so long as they are compatible with the overall goal.

The integration and consistentency of data across front, middle and back office was also interesting, and in particular the front office integration echos some of the things I have been saying about the need for analytics management and the management of front office data as part of the data management process, not something to be ignored in the hope it sorts itself out.

Posted by Brian Sentance | 5 March 2010 | 3:28 pm

Fund administrator or data distributor?

Just caught up with this article appeared on the A-Team website - Bloomberg is facing pressure from the industry with regards to users concerns about its initiative to make its codes freely available (see previous post Truly "Open" Bloomberg?). In the article, Max Woolfenden, managing director of FOW Tradedata, recognizes the potential of the BSYM website but advocates more progresses to be made in order to improve completeness of the data offered and in particular to clarify what exactly 'open' means.

According to A-Team, Bloomberg is also facing pressure with regards to a possible introduction of a new licensing structure for Service Provider Agreement (SPA) contracts for fund administration clients. Under the new system, fund administrators would be required 'to pay per security in each individual client portfolio', effectively changing the status of the fund manager to that of data re-distributor with all the cost increases that implies. It will be interesting to see where this heads - will the administrators simply pass the data costs through to their clients, absorb some costs as a competitive play or simply move away from using Bloomberg data?

Posted by Sara Verri | 23 February 2010 | 5:28 pm

More CEP Events

Sybase have acquired Aleri according to Finextra. It was less than a year ago when the complex event processing (“CEP”) vendors Aleri and Coral8 announced their merger (see press release); there was also a big buzz when Sybase announced a CEP capability based on Coral8 and Streambase decided to offer an Amnesty Program for Aleri-Coral8 Customers (see earlier post 'Merging in public is difficult...). And only a few months later, Microsoft announced that their CEP Orinoco (now integrated with SQL Server 2008 as StreamInsight) was heading to market (see post 'Microsoft CEP surfaces as 'Orinoco').

Another sign that CEP is moving more mainstream and that real-time everything is becoming more important? Or a good market for acquisitions?

Posted by Sara Verri | 4 February 2010 | 6:00 pm

"Cut and Paste" Valuation Services

You can talk about more robust modelling, more stringent scenario testing and even moving everything onto an exchange, but unless we move the principles of good data management (in my view: consistency, security and quality of all types of data) into the front office then we will continue to get front-office mis-marking as described in this article in the FT.

Thanks to Ralph Baxter from Cluster7 for highlighting this article for me and those of you interested in this topic of operational risk and spreadsheet mis-use should maybe go along to EuSpRiG this year, and maybe take a look at a paper Xenomorph presented at a previous conference.

Posted by Brian Sentance | 4 February 2010 | 9:49 am

Views on Fair Value...

Busy week last week for events in London, this time over at the Goodacre / Six Telekurs on Thursday morning. Guy Sears of the IMA was chair of the event, and the event did have a "buy-side" focus to it. Richard Newbury of Six Telekurs started the event and made the following points on the current state of regulation:

UCITS IV - Richard cited the stats that there are around 37,500 funds in the EU with average value of approximately $180M each as compared to only 8,000 funds in the US with average value over $1B. Richard said that such a proliferation of funds was costly and the more EU could standardise funds and their ability to be transacted everywhere in the EU the better.
Reg NMS - Richard took a little humorous dig at US regulators when he reminded us that Congress authorised the SEC to form a "National Markets System" in 1975 and so this had taken around 30 years to implement. Whilst Reg NMS is often compared to MiFID, he said that Reg NMS had led to consolidation in the US while obviously MiFID has led to fragmentation in the EU.
Hedge Funds - Both EU and US regulators are looking at the hedge fund industry. He mentioned the battle the UK was having with some of the (misguided?) regulation that the EU is trying to introduce with over 30,000 HF related jobs in London. The new regulation is likely to increase reporting requirements leading to more need for regular, standardised fair value reporting.
Credit Rating Agencies - Richard mentioned how there will be more ratings and more ratings types, and the regulation introduced to ensure the CRA do not fall into the conflict of interest trap.
Data Management - He mentioned the importance of data management within what is happening in the industry and noted how the profile of data management was on the increase.

Mike Jenkins of Ernst & Young tried his best to make the accountancy treatment of derivatives interesting and didn't do too bad an effort but I only took the following few notes from his talk:

Unlike US GAAP with FAS 157 there is no single standard Fair Value (FV) definition in IFRS, and unsurprisingly IASB are addressing this.
Mike spent some time mentioning Level 1(quoted), Level 2 (observable) and Level 3 (unobservable) pricing inputs for securites, taken from the IASB exposure draft ED/2009/5 (also see Rowe in earlier post)

Matthew Cox of BoNY Mellon Security Services then gave his presentation on the difficulties/challenges of providing a valuation service to their asset management clients:

His division often have a "2 hour" window to produce valuations for NAV reporting, often for a 12 midday valuation
Data exceptions for investigation went through the roof this year due to increased volatility (comment: didn't get chance to ask whether the validations set were "normalised" for market volatility i.e. a price movement threshold would not be fixed but rather be multiplied by a factor relating to recent volatility levels)
Matthew was very complimentary about the efforts his team put in to cope with this increase in data exceptions.
He mentioned how many of his clients of established "Fair Value Committees" over the past couple of years, comprised of staff from compliance, risk management, portfolio management etc.
Matthew mentioned the importance of time zones in valuation and the timeliness of data, with the availability of intraday CDS prices contrasting with bonds who price only from the evening close of the day before.

The panel debate was moderated by Guy Sears, and included the above speakers plus Nigel Reynolds from TD Waterhouse):

Matthew said that his division sometimes shared the "consensus" price from other clients when one client is looking for some guidance.
He mentioned that a key timeframe in establishing FV was establishing what is a "reasonable" time frame for sale of a security.
Nigel Cox said that "suspended stocks" had been a real issue over the past year, where the client "context" (position, situation etc) would very much determine what value a client would want assigned to a holding.
Guy Sears suggested that valuations should be provided with a confidence interval and not just as a single price
Mike of E&Y said that this is what full disclosure now requires, other memberrs of the panel suggested this was realistic but not what clients (humans?) expect to receive - they want a single number.
Guy wondered whether it was an issue that one entity might value an asset at a value X whilst another would value the liability at Y (not equal to X)
Mike of E&Y pointed out that this was an issue in that current accountancy rules allow a security to be reclassified from "fair value" pricing to "historic cost" basis - this discretion is being removed in future rule implementations
One member of the audience pointed out that Bloomberg, Reuters and Markit were all trying to extract more revenue from data used for valuation purposes.
Matthew advocated that the market needed more competition between niche data vendors such as Markit and SuperDerivatives to ensure innovation in service and more competitive pricing.
The audience asked Guy of the IMA whether the association should have offered more guidance on fair valuation process and best practice.
Guy said they have provided some, but he advocated that trade associations should not have opinions, since it was not healthy to have the asset management industry collectively herding towards the same valuations.

Well attended event with some good speakers, particularly Guy Sears as host was funny, knowledgeable and kept the other speakers on their toes. I would say the most interesting point was still that "opinions" form prices, opinions formed in the investment/funding "context" of the party with an interest in valuing a security - conceptually this seem to make the asset servicing companies a little uncomfortable since what they are contracted to do is to provide the "right" set of numbers by their clients. Human beings feel more comfortable fixating on a single number than a range of possible outcomes/results it would seem!...

Posted by Brian Sentance | 17 November 2009 | 10:48 am

It's in the news...

I went along to the Forum on News Analytics over in Canary Wharf on Monday evening, organised by Professor Gautam Mitra from OptiRisk / Carisma at Brunel University. We seem to be in the early days of transforming news articles into quantifiable/machine-readable data so that it can be processed automatically/systematically in trading and risk management. It was a good event with both vendors and practititioners attending so was reasonably balanced between vendor hype and the current state of market practice.

As background on what is meant by news analytics data, then for example you might count the number of news articles about a particular company and look at whether the quantity of news articles might be a predictor of some change in the company's stock price or volatility. Moving on from this simple approach (assuming that you are clever enough to be certain about what news is about what company), then you can then move towards assessing whether the news is negative, neutral or positive in sentiment about a company/stock.

The context here is about having the capability to automatically process/analyse any kind of text-based news story, not just those from research analysts that might be nicely tagged with such quantifiers of sentiment (see http://www.rixml.org/ on xml standards for analyst data). The way in which the meaning of the text is "quantified" uses some form of Natural Language Processing.

The event started with a brief talk by Dan di Bartolemeo of Northfield Information Services. I hadn't heard of him or his company before (maybe I should pay more attention!) but he seemed a very solid speaker with strong academic and practical background in investment management and modelling. He referenced a few academic papers (available via their web site) on news analytics, and how news analytics and implied volatility could provide better estimates of future volatility than implied volatility alone. He also made some good points about how investment "models" are calibrated to history and how such models need to adapt to "today" - he put it as "how are things different now from the past?" and put forward the idea of a framework for assessing and potentially modifying a model to respond to the "now" situation. He also suggested that the market can react very differently to "expected news" (having a range of investment "what ifs" planned for a known earnings announcement) as opposed to unexpected information (we are back into the realms of the Black Swan and the ultimate in uncertainty wisdom from Donald Runsfeld)

Armando Gonzalez of RavenPack then began by explaining how RavenPack had become involved in applying text analysis to finance (it seems the subject has its origins, like a lot of things, in the military). RavenPack seem to be highest profile quantified news vendor at the moment, and whilst Armando is obviously biassed towards pushing the concept that money can be made by adding quantified news data to trading models, he said that not many firms are as yet systematically processing news and most people are relying upon manual interpretation of the news they buy/use. Some of the studies Ravenpack have on market news and prices are very interesting, showing how a news event can take up to 20 mins before the market settles on a new "fair" price level for a stock. Additionally, and maybe an interesting reflection on human behaviour, was that in bull markets there are usually twice as many positive stories about companies than negative, but strikingly in a bear market there was still almost equal amounts of positive and negative news - so humans are basically optimists! (or delusional, or just plain greedy...take your pick!)

Mark Vreijling of Semlab followed Armando and suggested that a lot of their sales prospects understandably desire "proof" of the benefits of adding quantified news to trading, but this was a little ironic since most financial institutions have been paying to receive "raw" news for years, presumably because they perceive beneift from it. Mark also mentioned that the application of quantified news to risk management was a new but growing area for him and his colleagues.

Gurvinder Brar of Macquarie then went into some of the practicallities of quantifying and using news in automated trading. He suggested that you need to understand what is really "news" (containing information on something that has just happened) and what is merely an news "article" (like a "feature" in a magazine etc). Assessing relevance of news was also difficult and he added that setting a hierarchy of what kind of events are important to your trading was a key step in dealing with news data. Fundamentally he suggested that why wait for five days for analysts to publish their assessment of a market or company-specific event when you could react to the event in near real-time.

The event then went into "panel" mode where the following points came out:

Dan thought that a real challenge was integrating quantified news with all of the other relevant datasets (market data, but also reference data etc)
Armando picked up on Dan's point by giving the example news about Gillette which at one point was about Gillette the company but then on acquisition became news about the Gillette "brand" which became a part of Proctor and Gamble.
Dan said that a key problem with processing news was also understanding what news was simply ignored by the news wires i.e. we know what is being talked about, but what could have been talked about, why was it ignored and is it (even so) relevant to trading?
Mark and Armando said that the "context" for the news story was vital and that market expectations can turn many "negative" news stories into positive outcomes for trading e.g. the market likes bad news when it is not as "bad" as everyone thought.
Dan made a very interesting point about trading in terms of categorising trades as "want to" trades and "have to" trades. He gave the example of a trade being observed that seemingly has no news associated/prompting it - so does this mean the trade is occuring because somebody "has to" make the trade (a fund facing an welcome client redemption for example?) or because there has been some information leak to a market participant and such a participant "wants to" make a trade before the news becomes available to the market as a whole.
I think all of the panel members then collectively hesitated before answering the next question from the audience, with Microsoft having one of their "text search" R&D team (think Bing...) asking about news categorisation and quantification.
Dan also mentioned something that I have only recently become more aware of, which is that apart from major markets in the US, most exchanges world-wide do not publish whether a trade was a "buy" or "sell" trade (they just publish the price and transaction size). Obviously knowing the direction of the trade would be useful to any trading model, and Dan referred to this as wanting to know the "signed volume".
A member of the audience then asked whether most quantified news had been based on just the English language and the concensus was that most was based on English, but Natural Language Processing can be trained in other languages relatively easily. A few members of the panel pointed out that all languages change, even English, requiring constant retraining, and also that certain languages, countries and cultures added further complication to the recognition process.
The next question asked was whether the panel could outline the major areas that quantified news is applied in - the answer included intraday (but not quite real-time) trading, algorithmic execution, lower frequency portofolio rebalancing and in compliance/risk/market abuse detection.
A good debate ensued about whether "news" was provided by the official newswires or by the web itself. The panel (and audience) concensus seemed to favour the premise the news wires are the source of news and the web is a reflection/regurgitation of this news. That said, Gurvinder of Macquarie gave the nice counter example of the analysts/news wires not making much of the new Apple iPod, when looking at the web it was possible to see that the public were in contrast very enthusiastic about it.

Overall an interesting event. I think the application of "quantified news" to risk management is interesting - maths and financial theory is very interesting but markets are driven by people's behaviour and if "quantified news" can help us understand this better it has to help in avoiding (some!) of the future problems to be faced in the market.

Posted by Brian Sentance | 12 November 2009 | 12:06 am

Truly "Open" Bloomberg?

Interesting couple of articles from Inside Reference Data and Inside Market Data. The first is on Bloomberg making its codes freely available to all from its website http://bsym.bloomberg.com - given past standards-based attempts like ISINs falling short of providing the industry with unique and useful security IDs this looks to be a welcome addition. This seems to be a publicity "win" for Bloomberg, especially given rival Thomson Reuters has recently got some indifferent publicity with the EU over RIC licensing (see article). No prizes for anyone who thinks that Thomson Reuters will not respond in some way with regard to RIC usage, maybe giving us two working proprietary standards that go "open" - at least everyone would then be matching up Bloomberg Tickers and Reuters RICs in public rather behind closed doors - and maybe a good opportunity for a Wiki site to do the matching up?

The second relates to Bloomberg providing a open-source data distribution system called "The Platform", I presume as less expensive alternative to Reuters RMDS. Meanwhile Reuters is busying itself with the plans for its competitor to the Open Bloomberg terminal with "Project Utah". Obviously Bloomberg is comparatively unproven with regard to systems provision so this is a big change and will be very interesting to watch - from a technology point of view but also culturally since can Bloomberg turn away from thinking in "Terminals" all of the time?

Posted by Brian Sentance | 3 November 2009 | 3:52 pm

Integrated Data and Analytics Management

Xenomorph was one of the sponsors on the “Integrated Data Management” webcast last week, hosted by Inside Reference Data (audio recording available here). There were a number of interesting questions that arose from the Webinar.

One fundamental although somewhat academic question was "What is Integrated Data Management?". Certainly everyone seemed convinced that there would be less "Enterprise Data Management" (EDM) projects in future, given the expense, scope and scale of such projects. The concensus was that whilst the need for data management was better under stood across all financial institutions, data management projects would be bitten off in more manageable chunks by asset type, business function or division (so are silos back in fashion I ask myself?!). Coming back to the original question, I guess my slant on Integrated Data Management is that we are seeing more and more data management projects that have an integrated reference data and market data elements to them, primarily driven by the need to sort out data quality/completeness/depth for use within risk management (in light of the financial crisis).

Related to risk management, a topic I pushed was that given the origins of data management for STP/back office, and given the interest in low latency tick data management/analyis in the front office, there seems to be a market gap (particularly in the US?) on how to manage data such as IR/credit curves, volatility surfaces and other derived data sets. These data sets seem to fall into the gap between what is thought of as market data (primarily just prices) and what is reference data (IDs and terms & conditions). This is another area where a more integrated approach to data management would be beneficial, particularly in making all these datasets available for risk management.

Coming back to a "hobby-horse" of mine, then I also raised the issue that whilst it is fine to be doing great data management (high quality, complete datasets etc) what is the point if all of your data is ignored by the front office and Excel is used to download the data traders and risk managers need from Open Bloomberg. I think the management of unstructured data (spreadsheets, word docs etc) needs to be elevated as an issue since this (unfortunately?) is where most data resides currently, despite what we data management professionals like to think.

I also think that the principles of good data management (centralisation, quality and transparency) could apply to other things and not just raw "data", but what about centralised pricing and valuation, centralised curves and centralised scenarios for risk? Again what is the point of doing good data management if the ultimate "information" (e.g. a valuation) is done using poor quality data, with a complete lack of transparency over the data and model used.

A good question was asked about models, which was that given pricing models and their weaknesses have formed some part of the recent crisis, do we need more complex models. On having a few conversations about this and thought about it some more, then some would say it is complexity that got us into the crisis so this is the last thing we need. My view is that we do not necessarily need more complex pricing models and valuation techniques, but we certainly need more robust ones which does not necessarily imply more complexity. Coming back to a point raised by David Rowe previously, then I think all quants and risk managers should think about a "second means of valuation" for all the theoretical models they use, and that hedgeability (see recent post on pricing model validation) seems to be the common theme in producing more robust pricing models.

Posted by Brian Sentance | 21 October 2009 | 9:32 am

Pricing Model Validation: Mitigating Model Risk

I managed to catch some of the day yesterday at the "Pricing Model Validation: Mitigating Model Risk" conference. I thought it would be worthwhile going along since firstly the past 12-18 months have made model risk very topical (take a look at previous posts from Riskminds, the Modeller's Manifesto and Wilmott/Rowe).

Secondly more of our clients are looking at managing and centralising pricing models/curve calculators in addition to just managing the underlying data (see this Insight Investment client case study for a recent public example). I am calling this "Analytics Management" which is the business-focussed technology stack that combines pricing models/calculators/analytics with all of the "Data Management" underneath. But enough of my thinly-veiled positioning statements...and on with some of the (hopefully) useful content from the conference outlined below - maybe scan the headings in bold below for those talks of interest but I would particularly recommend the ones by Tanguy Dehapiot and Yuyal Millo...

Model Risk 2009 defining and forecasting. First speaker was Professor Phillip Sibbertsen of the University of Hannover on defining and measuring model risk. Phillip started by saying that "Model Risk" was a new category of risk within the confines of "Operational Risk", and that operational risk as defined by the regulators does not yet currently include the "model risk" of market risk and credit risk, nor the "model risk" of the operational risk model itself. (I am sure I could write that up better!...). Phillip put forward that model risk is not formally a "risk" since it has no probability distribution and that he suggested it should be thought of as "model uncertainty". He also clarified that model risk applies both at the large, portfolio scale (e.g. choice of VAR model etc) and at the smaller, instrument level scale (i.e. pricing of derivatives).

Additionally in terms of measuring model risk then he excluded human failure from model risk measurement since in his view this was difficult to quantify - this approach did not meet with the approval of some of the audience were questioning how this could be excluded from a practical point of view. Phillip's colleague, Corinna Luedtke, then presented some work they had done on calibrating different GARCH models to observed data and showing how even a poor model could produce reasonable forecasts of risk if the time period was short. The work was interesting but again the audience highlighted that the human choice (failure?) in choosing the set of models to try was part of "model risk" and should not be excluded from the definition of model risk.

Is a model accurate? Testing the implementation of a model. Second speaker was David Chevance, Head of Equity & FX Model Validation at Dresdner Kleinwort. David outlined the different sorts of model risk: mathematical errors, missing risk factors, divergence from industry practice, model inconsistencies and implementation risk. He then outlined the sources of these risks: bugs, approximations, numerical precision, numerical boundaries and limitations on numerical methods (e.g. Sobol numbers in high dimension monte-carlo simulations).

David said a key area to start with in validating a model implementation was the front-office documentation of the product, its inputs and payoffs, its pricing model but also details of calibration methods used/needed etc. He made the point here that the documentation can sometimes specify just the deal, but sometimes can express the pricing methodology and pricing parameters. The emphasis was on completeness, accuracy and making use of all of the information available in the documentation. Obviously the ability to review the code used to implement the model was also necessary.

He discussed the trade-offs between a simple validation approach in terms of speed and efficiency of resources against the more time-consuming, resource hungry but more accurate approach of full replication of the model. He also suggested that in choosing a method of validation it was important to balance resource demands against what is actually being validated: payoffs from a single trade, a type of pricing model or a family of financial products. Desired accuracy of the validation was also important, given the trade-off between accuracy and effort, and the fact that small bugs are much more common than large.He finally discussed model version control, the necessary discipline of documenting changes and regression tests for new models, and the regular cycle of model review. Overall it was an interesting talk with a good practical focus.

Practical aspects of valuation model control process. One of the most entertaining and interesting speakers of the day was Tanguy Dehapiot, Head of Validation and Valuation, Group Risk Management at BNP Paribas. He started by referring to a few documents "Supervisory guidance for assessing banks’ financial instrument fair value practices", April 2009 (BCBS 153) which was then implemented within “Enhancement to the Basel II framework” (BCBS 157). The first part of his presentation was around these documents and what the regulators expect to be in place, so I guess the best approach is to read them (the BCBS 153 document content is only 12 pages long, quite short for a regulator!)

Tanguy pointed out that in his view "Mark to Market" and "Mark to Model" are often misleading as both are often required. He prefers the term "Valuation Methodology". He proposed four valuation modes: Direct Price Quotation, Use of Similar Instruments, Risk Replication, Expected Uncertain Cashflows (NPV) and categorised a useful hierarchy/matrix of which financial products fit into which valuation mode and for what purposes. Within model risk, he split off judgemental errors (choice of model etc) as part of market risk and credit risk and operational errors (model implementation and coding) as more definable and avoidable parts of operational risk.

He had some interesting slants on data, saying that he had been surprised that even getting all of the static data necessary to price simpler instruments like bonds had proven difficult. He outlined how model parameters are often stored across a variety of systems (curve definitions in one place, pricing methodology somewhere else) implying to me that this is sometimes difficult to pull together and needs some centralisation to improve transparency around this.

His opinion on market parameters (both observed prices and derived data such as implied volatility surfaces) were often stored in a larger central database but warned that this market parameter database needs to be reviewed as part of the model validation process since some of its data is derived (i.e. calculated, maybe using a model!) and as such should not be taken as perfect for all time and for all purposes. He said that it was important to categorise the origin of data and suggested the following types:

Quoted on an active exchange
Actual private transaction in an active market
Tradable broker quotes
Consensus prices from market makers
Non-binding indicative prices from market makers
Counterparty valuation, collateral valuation
Actual transactions in inactive market

Tanguy proposed that there should a valuation matrix for each instrument, where there might a different valuation methodology used for end of day valuation verses intraday, for risk or for trading, for pricing individually or within a portfolio reval. I guess here the rational is appropriateness, efficiency and transparency about what needs to used when. He also added that he disliked the term "Model Validation" since it seemed to imply that a model was "valid" and preferred "Model Approval" to cover the decision to use a model and "Model Review" to cover model analysis. He said he found managing the "stock" of existing models (and keeping up with when to review them) more difficult than managing the "flow" of new models and products.

Overall Tanguy was a very interesting and funny speaker with lots of practical insights and a fair amount of opinion thrown in, which is always good in my view.

The usefulness of inaccurate models: Financial risk management "in the wild". This talk was given by Dr Yuval Millo of the London School of Economics and he focussed on the evolution of the use of the Black Scholes Merton (B-S-M) model at the CBOE and how the model came to be the means by which the whole options market "communicated". Yuyal is a social scientist and prefaced his talk by stating that "Social Sciences are good at predicting the past"

First thing I didn't know (amongst the many things I do not know...) is that the B-S model was not published until a couple of weeks after the CBOE started trading stock options in April1973. Yuyal said that initially the B-S-M derived prices were not accurate at all (around 25% off the market price on CBOE) and that the model was based on assumptions that plainly were not the case on the exchange (only calls available, no short selling, no continuous trading). The model was used by local Chicago trading firms and the story goes that Fischer Black sold large paper "sheets" of option pricing matrices to these traders (there being no calculators/PCs/mobiles around at the time).

As the markets developed, larger East Coast banks entered the market with stocks being held and traded in New York and options being traded in Chicago, so trading became geographically dispersed. This started the need for "early morning meetings" to discuss the market and the B-S-M model and its parameters became the "lingua franca" or means of communication of options market participants.

He described the first years of the Options Clearing Corporation (OCC) which was set up to ensure that the financial obligations of options and buyers were met. Around 1979-80 the OCC worked overnight to calculate margin requirements, based on the (now?) arcane idea that different margin amounts should be associated with different option strategies (straddles, butterflies etc) and the job of the OCC was to take a portfolio of Option and optimise which combination of strategies would minimise the margin required for the whole portfolio. He said that there were disputes between traders and the OCC around margin levels and difficulties for the SEC with updating their Net Capital Rules as each new option strategy was created. Eventually, the OCC adopted the B-S-M model and implied volatility as the means of calculating margin against market value which enabled them to move away from the operational difficulty of strategy optimisation.

So the B-S-M became the way in which traders communicated about the market but also the model became vital operationally within clearing for the market. By 1987 B-S-M had become the de-facto standard for the market, with the model driving the market in turn driving use of the model. During the Oct '87 crash the model proved to be very innaccurate but the use of the model did not diminish - maybe pschologically the market participants needed a model (even a wrong model) to make communication easier.

I found this talk very interesting and members of the audience asked whether any similar analysis was going to be done on the Gaussian Copula model used to price CDOs. Yuyal said that one of his colleagues was undertaking this research currently. Given that he seemed to be very positive about the use of the B-S-M model within options markets I asked whether he had any opinions on Taleb's criticism of fiancial engineers and modelling. Yuyal said that he and Nassim were friends and agreed to disagree on certain topics...

Stress testing modelling parameters. Next up was Peirpaolo Montana, Head of Model Validation at West LB. Having joined the finance industry out of a career in mathematics and then at a regulator, Pierpaulo began by saying that back in the heady days of 2004 the banks thought that their own risk management systems and practices were well ahead of the regulators. He said that in light of the crisis this proved not to be the case but he now feels that this is now more evenly balanced (not sure I would agree, still lots of catchin to do for some institutions I would suggest).

He said that whilst regulators require the validation of risk models and pricing models, and that stress testing of a portfolio is required, that the stress testing of a pricing model is not a requirement and has received much less attention and in his view was not done to much degree before 2007. His point here was that pricing models should work under stress too, otherwise they are a weak foundation for building other risk measures such as stressed VAR.

Whilst focussing on pricing models, he mentioned that risk models also need to be carefully chosen and appropriate to the institution and the types of trading activities it undertakes. As an example he put forward that a simple VAR calculator might be appropriate for a long only equity fund but completely innappropriate for a relative value portfolio.

He said that stress testing had recently received much more attention as a risk management tool and cited the BIS document "Revisions to the Basel II market risk framework" where stressed VAR is introduced as part of the regulatory capital charge calculation. He also mentioned that in order to avoid "standard model" treatment of complex securitised products an institution must be able to demonstrate that its VAR model can cope with these products under times of market stress.

Pierpaulo then described the stress testing of base correlation in CDO pricing, and how even moving the base correlation from its usual level of 70% to 99% would not have predicted the valuations observed in the recent crisis. In this way he says that stress testing of models can detect implementation problems and some model weaknesses, but it cannot assist in coping with structural breaks in the market. He also discussed how the B-S-M model is used everywhere (even places it should not really be valid for) since it is a robust model based on the no-arbitrage hypothesis - in contrast the CDO base correlation and other models are not so robust since they are not arbitrage free.

(end of post!)

Posted by Brian Sentance | 18 September 2009 | 4:30 pm

Heavyweight Data Management...

...I am very concerned that I have previously missed an important requirement for data management solutions - a heavweight one judging by this great discussion on one of the Microsoft forums.

Posted by Brian Sentance | 17 July 2009 | 7:17 am

Microsoft CEP Surfaces as "Orinoco"

Seems like Microsoft have now gone public on the Microsoft TechEd site that they have a Complex Event Processing (CEP) engine that will be coming to market shortly (see MagmaSystems blog post ). One of my colleagues Mark Woodgate attended a briefing event at Microsoft for this technology back in February this year - here's an extract from some internal notes that Mark made back then:

"Microsoft CEP is very similar to StreamBase conceptually (and not unsurprisingly), in the sense that there are adapters and streams and how you merge and split them via some kind of query language is the same. However, StreamBase uses the StreamSQL which as we have seen is SQL-like in syntax but Microsoft CEP uses LINQ and .NET and although conceptually it is doing the same thing, it does not look the same. StreamBase’s argument was you can be an SQL programmer to use it and don’t need lower-level like .NET; however, it’s not SQL really as it has all these ‘extensions’ you have to learn so using .NET might look more tricky but in fact it makes sense. They don’t have a sexy GUI yet for designing CEP applications like StreamBase but it will be done in Visual Studio 2008.

Currently, you build various assemblies (I/O adapters, queries and functions) and then bolt them all together, called ‘binding’ by command line tool. You then deploy the application onto one or more machines using another tool so it’s a manual process right now. They are aware this needs to be made easier and more visual. They are allowing other libraries to be bolted in via the various SDKs so it’s pretty open and flexible. It works well with HPC and clusters/grids (or so they say) and of course can be used with SQL Server. The CEP engine also has a web interface based on SOAP so at least non-Windows based systems can talk to it"

The release of this technology will be an interesting addition to the CEP market and to the Microsoft technology stack in general. Assuming performance is at credible levels (i.e. not necessarily leading but not appalling either) it will certainly bring both technical and commercial pressure to bare on existing CEP vendors (see earlier post on Aleri/Coral8) and has the potential to broaden the usage of CEP. Obviously Linux-Lovers (sorry, I didn't mean to be personal...) will not agree with this, but Microsoft is putting together an interesting stack of technology when you see this CEP engine, Microsoft HPC and Microsoft Velocity coming together under .NET.

Posted by Brian Sentance | 14 May 2009 | 4:13 pm

Data Quality and the Future of Risk

A new survey from the Economist Intelligence Unit (sponsored by SAS) of over 300 financial institutions world-side has put data quality and availability as a key issue to be resolved if risk management is to be fit for purpose following the financial crisis:

"Culture, expertise and data are weak points in current risk management"

A summary of the survey report is available here.

Posted by Brian Sentance | 8 May 2009 | 9:31 am

« Back to recent posts

Contact us if you have comments. All rights reserved. Trademarks, copyright and legal. Whole site ©1995-2011 Xenomorph Software Ltd. Registered in England and Wales, Reg no: 03235432, Reg at: Waverly House, 7-12 Noel St, London, W1F 8GQ. VAT no: 672584016 - sitemap

Categories

Posts categorized "Data Management"