Determining the facts about an organisation’s data is essential to bring about a step change in data governance and to ensure compliance with the Basel Committee for Banking Supervision BCBS 239, Principles for Effective Risk Data Aggregation and Risk Reporting (PERDARR).
In a short anonymised case study about a bank credit risk modelling group, Jon Asprey, vice president strategic consulting at Harte-Hanks Trillium Software, told the recent FIMA Europe conference in London: “The regulator is guiding, cajoling, and moving you into the right place and really telling you the foundational capabilities, skills and processes they would expect you to have in place to manage your risk data effectively.”
Facts about the data
“To be compliant and to demonstrate the capabilities required by BCBS, all the data management and data quality components that you need to put in place root themselves in the analysis of facts about your data.”
Asprey added: “When you’re trying to establish the culture of governance I often say it’s about trying to get people to be data-aware – not data gurus but conscious of the part that data plays in the business.”
These are some of the high-level steps that need to be taken:
- Create the knowledge-base: what do we currently understand about our data? That means collating that, documenting it, making it accessible and visible and searchable to the people who need to know.
- Encourage and drive collaboration. Organisations are silo’d but often data-driven processes have a number of hand-offs: they may start in operations or relationship management, move into IT and warehousing and then risk and finance and maybe customer support – so there are many hand offs so you need to drive that collaboration.
- Measure the business impact and value.
- Determine the process for taking action: who’s responsible? What happens?
There are three tactics for success:
- Connect the stakeholders.
- Link the data to the business process so as to be able to assess the impact of the problems and the priorities. “You’ll find 101 problems – the challenge is, which are the important problems and which are the ones you try to fix first?” Asprey said.
- Connect governance and data quality management to a measurable improvement in the business. Money is spent on tools and resource are devoted to this – but how do we record the payback?
Case study: The mystery of code 15
In the case study example given by Asprey, his firm was working for the credit risk modelling group of a retail/commercial bank. The engagement objective was to identify and interview the stakeholders and build a set of data definitions (what does the data mean, what are the expected values, what’s the English description, what are good and bad values?).
Each of the bank’s customer was assigned a ‘risk category’.
- The source system owner said that codes 1 to 14 were valid and that there shouldn’t be any blanks.
- The transformation layer (ie, the data warehousing team) said that they have a script they run so that if there are any blanks, they are replaced with code 1.
- The model developers said they use all codes that are in the system, with no filters or checks applied.
It transpired that 125,081 customer files had been assigned a code 15 – a code that it had been said didn’t exist, wasn’t valid and shouldn’t be used.
“We used this fact as a way to get those people around the table and say, ‘Well, you’ve all told us a part of the story but when we analysed the data this is what we see – so let’s try to understand actually what’s going on,’” Asprey said.
- Then the system owner said code 15 was a redundant code and that it shouldn’t be being used for any new data. A change request was in place that had not been implemented to take it out of the system and to stop it being used.
- The data warehousing people said code 15 was a known issue and a manual recoding process was in place to update the code 15s to whatever code it should be.
- And the model developers said they deploy a “simplifying assumption”: if the data doesn’t look right they put something else in its place, such as an average value or a default value.
“The interesting thing was the head of portfolio analytics who owned the model developers was on a mission to remove simplifying assumptions,” Asprey said. “The regulator was saying you shouldn’t be doing them.”
As one of the senior stakeholders said during the discussions, “What this is doing is turning a data story about code 15 into a business story of people doing extra work and having things that are inaccurate and having invalid data.”
“When we started the process, the model-development team were very uncooperative, “ Asprey said. “They didn’t want everybody to know that they were using data that wasn’t very good, they didn’t want people to know that simplifying assumptions were in place – it was just not a healthy environment.”
Now, the stakeholders at the bank are fully on board, they understand that their models are better, they can give better mitigation and explanation to the regulators, and they are in a much more transparent environment.
“But the interesting thing is, by unearthing the facts we were able to get the dialogue. In terms of business performance the accuracy of the data was improved, the productivity of the agents was improved, the number of adjustments to the data and manual work was reduced, and ultimately for the model development guys, they had better model performance, fewer simplifying assumptions, and a better story for the regulator,” Asprey said.
“That factual analysis was the key driver in getting all these guys around the table, understanding the impact, and get agreement on what the problem was and what plan of action we needed to put in place.”