Categories
Data Governance
Data Strategy
I joined DAMA UK to connect and exchange ideas with other data specialists. The DAMA EMEA (Europe, Middle East and Africa) Chapters conference was an opportunity to turbo-charge that goal. When some of my committee colleagues suggested I go to present the findings of the UK chapter’s survey, I jumped at the chance.
This year’s meeting was a hybrid event, hosted in Bologna, with about 1000 virtual and 200 in-person participants, representing 60 countries, hundreds of organisations, and all kinds of data, technology and business functions.
The first day covered a wealth of big ideas in data practice. They were organised into three parallel streams, and the only grumble I heard was that everybody wanted to go to everything at the same time!
I started with a panel discussion featuring experts in the health sector, each effecting deep, and genuinely inspiring, transformation at very different stages of their organisational lifecycles.
I dropped into a great talk on socialising AI into our organisations: how to engage business-experts, especially the decision-makers, to frame the questions; and how to find ‘friendly’ ways to get them hands-on with the technology itself.
I went to see my friend, Ole, who literally wrote the book on Data Catalogues, walk us through the synergy with knowledge graph (spoiler: it doesn’t get you off the hook of data management; you have to organise your data first to unlock the power of any catalogue).
Then another panel discussion, looking at the democratisation of environmental data. Then another presentation, on what to expect from the impending EU AI Act.
By the end of the day I needed a lie down in a darkened room just to digest the dozens of new ideas (and quite a lot of tiramisu) I’d already consumed.
Day 2 brought the focus, largely, onto the work of the DAMA chapters themselves.
We heard from groups translating the DMBoK into their native languages, and the significance of that work in growing awareness of data management beyond the professional community. My mind was blown by hearing that the Finnish language has a single word meaning ‘data’ AND ‘information’ AND ‘knowledge’; that’s a challenge!
We had an exhilarating presentation from our youngest chapter, DAMA Egypt, who have really hit the ground at a sprint. It’s just two months since they were officially recognised by DAMA International, and already they have set up study groups, schools outreach, and are engaging with regulators to offer expert input into Egypt’s singular data protection laws.
I’ll admit, I was proud to hear my committee colleagues, Nigel Turner and Mark Humphries talk us through how DAMA UK has grown into far-and-away the biggest chapter in the EMEA region, and how we are having to reorganise our operations and provision to keep pace with our growing membership.
I spent the first half of my afternoon in a fantastic workshop, exploring the implications for data management of AI law. While it focused on the EU AI Act, this exercise gave a structure to help unpack data management’s role in the ethics and effectiveness of AI, as well as in the international race to regulate. If anyone would be interested in a DAMA UK version, I would love to hear from you!
Finally, I dropped back into the DAMA coordination meeting in time for a session on defining our shared objectives for the coming year. The UK delegates split our focus across themes: I chipped in on curriculum development and engagement with education; Mark on an ambitious vision of data maturity assessment at national scales; and Nigel on plans for the next conference…
Yes, there will be a next DAMA EMEA Conference, although that’s really as much detail as we have for now. It might be in 2024, or maybe 2025. We don’t know where, or by which chapter it will be hosted; possibly France or possibly even the UK. What I can say with confidence is that it will be worth going to.
Most strikingly, in its content and atmosphere, this was a community event: very different from the huge, commercially-oriented data conference we see filling our LinkedIn feeds and London’s exhibition centres.
Everyone, including the corporate sponsors, came willing to participate as equals. I arrived with just a handful of social media contacts, and left having established some wonderful friendships across continents!
As I said in my talk - more than anything, in our mission statement, and even on our logo - we are a community, working together and learning from each other to promote the value of our profession. The rewards are best – the inspiration and the empowerment – when we engage as a community: by showing up and chipping in at webinars; or making the effort to get to a regional meet-up; or trekking halfway round Italy to be inundated with learning and desserts.
As we roll into a new year, I really encourage you to think how you might engage a little bit more, and get a lot more out of your DAMA UK membership in 2024.
In a world increasingly reliant on data, access to high-quality data and information it is crucial for driving innovation, economic growth, and societal development. The Department for Science Innovation and Technology (DSIT) is carrying out a review of what public sector data has the potential to unlock most value to both industry and the wider economy. Put in simple terms, is there more data government could share and if so, what data is it?
DAMA UK joined forces worked with DSIT canvassing our DAMA UK members. We held two insightful roundtable discussions. These sessions aimed to explore views from our members on ways of increasing access to public sector data. This builds on our members previous comments on the government’s National Data Strategy. DSIT are also working with other organisations and carrying out interviews with experts from many sectors to inform this review.
Why do the review now?
The Pro-innovation Regulation of Technologies Review: Digital Technologiesfound thatgovernment and broader public sector bodies hold significant data but that the ability of the private sector to access this data is inconsistent and fragmented.
DSIT is working with Centre for Digital and Data Office (CDDO). You may have seen the work from the CDDO on the data market place. This is looking to share government data across other government departments. It is looking at Essential Shared Data Assets (ESDA’s). Allowing government to share the most important data across departments. By DSIT and CDDO working together, it covers both government and private sector requirements for data.
What did our members say?
Our discussions covered the current public sector data sharing ecosystem. With our members from various sectors sharing their experiences, providing valuable insights into the state of data sharing in the public sector.
Value was a key question, and value for whom? Understanding the stakeholders and beneficiaries of data is fundamental to harnessing its potential. This value is more than monetary, it can be to society or to the environment. The value of geospatial data, particularly core reference data like addresses, is indisputable. However, its availability for free to the public sector and must be paid for by everyone else, can create power imbalances, limiting its potential for businesses and for innovation. This issue aligns with the European Open Data Directive, which promotes open access to valuable datasets which are largely geospatial and core reference data.
We covered whetherFreedom of Information requests (FOI) hold insights into the data people required from government. And whether Large Language Models (LLMs) creating shadow data, could potentially eliminate the need for official government data sources. We decided shadow data was good for some things, but there would always be a need for government data.
Enhancing the discoverability of data was another challenge. Some DAMA UK members had not come across data.gov.uk or the Defra Data Services Platform that hold a wealth of data held within government.
Artificial Intelligence (AI) & Machine Learning (ML) have become the focus of global news headlines over the past months. Pessimists fear that the growth of AI/ ML poses a serious threat to the future of humanity, invoking Terminator style doomsday scenarios. Optimists claim AI/ML can be the saviour of humankind, a vital tool in helping us identify and avoid impending future problems and disasters, often before we are aware of them.
The reality is, as always, somewhere between the two. Like any new set of technologies AI/ML has the potential to benefit us all if applied ethically and intelligently. A growing library of use cases is already beginning to appear which show how AI/ML can identify and help to create new opportunities and resolve problems in areas such as government, retail, banking, insurance, manufacturing, travel etc. But if used wrongly or for morally questionable purposes such as the dissemination of political misinformation AI/ML could cause intended or unintended harm. So what can we do as data management specialists to play our part to ensure that AI/ML is a force for good, and not a force for evil?
One key way is to recognise and promote the fact that AI/ML, like any set of technologies that relies on data is only as good as the data it is given to work with. However carefully the algorithms that drive AI/ML are constructed and applied, they will invariably produce false outcomes if the source data is not a true reflection of the reality that data is supposed to represent. Put simply, AI/ML critically relies on good data quality. Feeding AI/ML with inaccurate and incomplete data inevitably results in it generating outcomes, decisions and actions that are inaccurate, unreliable, misleading and potentially downright dangerous.
To use a simple analogous example, AI/ML could be tasked with solving a jigsaw puzzle. It would have to be taught the rules of how a jigsaw works, in particular recognising that a complete picture has to be built from the 1,000 jigsaw pieces it is presented with. As we would do, it could start by identifying the four corner pieces and remaining frame pieces identified by having one straight edge. It could then assemble the pieces to form the complete frame, and progress its completion from there. If all 1,000 pieces were present and correct, this is an achievable task. But what if some jigsaw pieces were missing, some pieces were duplicates of other pieces, and other pieces were from a totally different jigsaw? Suddenly the task becomes much harder and the outcome less certain and reliable. A similar process also applies to data. If pieces of required data are missing, duplicated or invalid, AI/ML may struggle to create the finished picture intended. Worse still, it could generate a different picture altogether.
So getting data quality right is a ‘must have’ for effective AI/ML. Yet this is not the reality for many organisations who are using or thinking of applying AI/ML. A recent Capgemini survey found that 72% of business and technology executives stated that the biggest barrier to implementing AI/ML and data analytics in their businesses was fragmented and poor quality data. [1] And it must be stressed that this is not just a problem for AI/ML or data analytics. Poor data quality continues to hurt business profitability, efficiency, productivity and decision making. A 2023 survey by Drexel University & Precisely found that poor data quality is ‘pervasive’ in most organisations with 66% of respondents rating the quality of their data as ‘average, low or very low’ [2]. So how do you spot poor quality data? Its main symptoms are:
Moreover the relationship between AI/ML and data quality is not a one way street. Whereas effective AI/ML depends on good data quality, AI/ML can itself be used to help to solve data quality problems. Many data quality software vendors have recognised this and are already embedding AI/ML functionality into their toolsets. AI/ML can help address existing data quality problems and proactively prevent future problems by:
Poor data quality is indeed an enemy of AI/ML, but using AI/ML approaches and capabilities to identify and tackle data quality problems is a clear win-win. Better data quality will make AI/ML more effective and useful; AI/ML can help to create the better data it needs to improve its business value. Using the techniques of data quality and AI/ML in tandem can bring mutual benefit and better business outcomes. Whereas today AI/ML and data quality can often be presented as enemies, they can and should become the best of friends.
[1] Cited in “Intelligent MDM, 2nd Informatica Special Edition”, Lawrence C. Miller, 2023
[2] “2023 Data Integrity Trends and Insights Report”, LeBow College of Business, Brexel University and Precisely, 2023
Embracing Change: A New Era for DAMA UK and the Data Profession in the UK
I’m Lisa Allen, I have the immense pleasure to take on the role of Chair of DAMA UK and I would like to welcome our new committee and extend my heartfelt thanks to the outgoing members. As the Chair of DAMA UK, I am excited to embark on this journey with fresh faces and experienced hands alike, because the next two years are poised to be pivotal for DAMA UK and for the data profession.
First and foremost, I want to express my deep appreciation for the dedication and hard work of our outgoing committee members. Your tireless efforts saw DAMA UK's growth and success. Your commitment to advancing the data profession has not gone unnoticed, and we owe you a debt of gratitude for your invaluable contributions so a heartfelt thanks to Nicola Askham, Ian Chapman, Aktar Ali, Andrew Lunt, Phil Jones and James Tucker.
As we bid farewell to some, we also warmly welcome the new committee. This year we saw more applications to join the committee than ever before. We are grateful to all those who put themselves forward. Our new committee members come from diverse backgrounds and bring a wealth of experience in the data field. They were voted on by you or co-opted to join. Their enthusiasm and passion for data management is truly inspiring, and I have no doubt we will make significant contributions to DAMA UK's mission.
Looking ahead, the next two years are shaping up to be incredibly important for DAMA UK. Our organisation has grown significantly in recent years and across a diverse range of sectors, both public and private. This reflects the increasing recognition of the importance of the data profession. In an era where data drives decisions across industries, our role as data management professionals has never been more critical. We must continue to evolve and adapt to meet the changing needs of our members and the data community as a whole.
During the next two years, we will focus on several key priorities:
1. Membership Growth: We aim to expand our vibrant community, welcoming more professionals into our midst. We’d like to have 3000 members by Dec 2025 to show the strength there is in our profession and continue to learn from each other.
2. Elevating Industry Standards: Our commitment to raising the bar for our industry includes fostering the growth of certified data professionals and advocating for the recognition of their expertise akin to other professions like accountancy and medicine.
3. Enhanced Member Support: We recognise the importance of providing unparalleled support to our members, ensuring they have the guidance and resources they need to excel in their careers.
4. Regular Offerings Review: To stay ahead in the evolving data landscape, we will regularly review and update our offerings to provide the most relevant and up-to-date information, training programs, and resources.
5. International Collaboration: We value collaboration and knowledge-sharing globally. We seek to foster strong partnerships with other DAMA chapters worldwide to exchange best practices and drive the advancement of data management as a profession.
I am excited about the journey ahead and the positive changes we can make together.
In closing, I want to thank each and every one of you, our members, for your continued support. Your passion for data management fuels our collective success, and I am confident that with your dedication, the data community in the UK will continue to grow and to thrive.
Here's to a promising future for DAMA UK and the data profession at large. Let's embrace the opportunities that lie ahead and work together to make a lasting impact.
If you’d like to see who is on the committee, you'll find all the great people here.
Lisa Allen
Chair, DAMA UK Committee
Framework for Success: How to transform your organisation for a Data-Driven future - Dave Knifton
Jenny Andrew, DAMA UK Committee member and head of data at the Chartered Society of Physiotherapy, hosted an entertaining presentation by Dave Knifton, founder of The Data Gym. The webinar was the first in DAMA’s ‘Unleash Your Data’ series with an overarching aim to boost data-led innovation, decision-making, efficiency and agility at organisations.
We work in distinctly different organisations, but they all have common characteristics. Turning data into a useful resource rather than viewing it as simply a series of problems is a mission that unites all workplaces. But how do you even start to make strides towards a data-driven future that is better for employees and customers alike?
During the webinar, Dave explored the foundations of a data-centric strategy that can transform your vision into a deliverable reality. This results in stakeholders having a deep understanding of:
Dave recognises that every organisation is different, and used his presentation to frame these disparities using two types of hypothetical businesses.
The first is an ‘unenlightened’ organisation. Data activities, processes and professionals are found under one roof. However, these firms don’t embody a complete data strategy. As a consequence they run the risk of simply ‘tinkering’ with data rather than extracting value from it.
The second type of organisation sits at the other end of the spectrum. It’s a ‘data-centric’ entity that functions differently to its unenlightened counterpart. It does innovative things with its data that move the business along to new areas that might not previously have been considered.
Despite these obvious differences, Dave argues that common approaches to data management are the only feasible way forward if goals are to be met, and standards and regulations developed and applied across sectors.
Unleashing the power and value of data creates a wealth of opportunities. However, there is much work to do before organisations reach that point. This begins with determining where your organisation is on its journey to becoming data-centric.
Data-centric organisations have many key features in common. Largely, these can be divided into three areas, which Dave examined:
Revenue generation, including data-informed decision-making. This begins with a strong data culture, with employees understanding how to interact with data and the impact it can make. Too many businesses are ‘data inert’ and focus solely on dealing with problematic processes; rather than fixing them but also using data to innovate for growth.
Operational efficiencies - more agile, cost-effective and sustainable (in both senses of the word). Data literacy is key here, with employees empowered to make evidence-based decisions when using data while having a clear view of intended outcomes. In other words, the whole organisation is built on a foundation of ‘data decision-making by design’. Teams are connected by data; it isn’t hidden in silos; and there’s a line of sight between data input and results, enabling the measurement of positive impact. Automation has a role to play in simplifying and augmenting processes, and thereby achieving operational efficiencies (although the debate about human vs. machine was not a central theme of the presentation).
Reduced risk of data mismanagement e.g. GDPR non-compliance, and not using data for its intended purpose. Everyone should care that the data being used has the correct characteristics, as Dave calls them, to produce a well-oiled machine of data-controlled outcomes. It also means having a streamlined data estate. This provides legible, understandable datasets that are fit for purpose. Importantly, your firm needs assurance in business-critical matters such as compliance. A data-centric approach will provide this, potentially through a Data Policy Framework.
Achieving success in those three areas described by Dave will afford your organisation ‘sure-footed agility’. But the positive outcomes will only be achieved if organisations realign around data.
To do that, they must design and implement a Data Assurance Framework that articulates solutions to aspects of data culture, data literacy, management & governance, skills, processes and outcomes.
This means implementing accessible templates that can be replicated and used across the business. This is true even for teams and individuals who don’t really care about data or how it is employed. The templates should be housed in a toolkit or ‘patterns store’ that all employees can access:
Ultimately, Dave believes the framework for a data-centric strategy can relieve the tension that lies between often-misplaced ‘delivery at pace’ that is intended to satisfy stakeholder demands, and a new, considered mindset of ‘delivery principles’ that breed lasting success.
These principles must be focused on sustainable, sensible outcomes that are driven by improved data management and performance. In this way, data governance, management and business architecture inter-relate harmoniously to enable better processes.
We’re keen to hear about your own experiences of building a data-centric organisation. Does it align with Dave’s Data Assurance Framework? Or did you take a different approach? Contact DAMA with your thoughts.
Every enterprise is focused on transformation to optimise the customer experience (CX). You’ve probably noticed, however, that senior leaders don’t always recognise such a change in strategy must always be underpinned by valuable, structured data. Without it there will be no transformation.
Scott’s presentation was a call-to-arms for data management professionals, summarising his personal mission to help our industry make the case for ‘better data’. It’s something you’ve probably sought advice about. You may even be building a business case for data management right now.
Scott recognises it’s rarely as simple as putting together a plan, then presenting it to the senior leadership stakeholders. This can be due to a breakdown in communication (ours is a complex field!) or perhaps a lack of understanding that data is vital for a shift to tech-driven operations. As Scott states: “Digital needs data, and data needs data management.”
Transformational CX has many components. Your organisation could be seeking to shift its model to put the customer at the heart of everything it does. Alternatively, the goal might be a switch to direct-to-consumer commerce; a greater focus on self-service; or the need to reassure the target market about aspects of data privacy and consent.
Of course, data underpins all of these strategic imperatives. Yet it isn’t always available to the required standard, or in a structure that is fit for purpose. Scott says the first step to better CX is defining master data - “All of the data in charge of your business” - in a way that’s easy for busy or baffled boardroom folk to understand. He reveals a good way to encapsulate this is “The 4 Cs” of master data:
In the simplest terms (for non-data experts), data organised around The 4Cs will tell your business who, what and where it relates to, and why it’s unique.
Taking a focus on current macro-level CX trends is key to show no modern business process will work properly without data management. These are the commercial trends the CMO, CTO and CEO and others will likely be aware of already. If you can frame the need for quality, structured master data in these terms it will make your case stronger.
Here are five trends DAMA has identified as being in play at many organisations:
As Scott says: “Transformation around these and other trends won’t work if you don’t have the data to back it up. That means having a 360-degree view, tying together disparate data sources across platforms. Making it available to the entire enterprise takes a lot of data structuring behind the scenes.”
A final piece of the jigsaw when setting out your case is to describe where your organisation is on its transformation journey. Scott identified three stages:
The consistent requirement to move from “silo to centric to network”, as Scott puts it, are authentic identity - trusted data - and a common data structure.
If you’re still unsure how to make the case for data management, Scott advocates a focus on the business benefits.
He uses a framework that he labels “the eight ‘-ates’” to lay out a transformational roadmap, complete with tangible outcomes to make a play for investment in better data: relate (connect with stakeholders); validate (present evidence); integrate (pull sources together); communicate (in clear terms); aggregate (analyse and report); interoperate (connect with third-parties); evaluate (what you need/want); and circulate (data flow drives value).
We’re keen to hear your own tales of presenting a data management strategy to senior leaders. What were the barriers, and what helped you to succeed?
If your organisation is seeking to appoint a data scientist, or it is your current job title, our recent online panel discussion about all aspects of the role offered lots of food for thought.
The webinar, hosted by our Committee Member Nicola Askham, brought together four data science experts to share their views:
The lively discussion was a whistle-stop tour of many aspects of data science: from trying to nail down a definition, to helping firms to identify the optimal time to create and fill the role.
While many organisations are in agreement about the growing importance of data scientists, the role doesn’t seem to have a single definition. The Government’s own career guidance website reveals how tricky it is to apply a standard job description: “Data science is a broad and fast-moving field spanning maths, statistics, software engineering and communications.”
This is a problem for organisations. With more than one million data scientists already employed at UK businesses by mid-2021 it’s vital for firms to understand what these data experts can do for their business. Organisations often rush to recruit them without first thinking how they’ll really help the business - and what needs to be in place before they arrive so that they can hit the ground running.
The panel began by discussing some of the misconceptions about the role and how this issue affects current performance. Participants pointed to a “broad church” of inter-related disciplines mentioned in job descriptions, before attempting their own definitions.
Marisa believed the data science role exists to use computer algorithms to extract and apply insights from data, informing real-world decisions. Shorful explained the scientist component describes using data to generate and test hypotheses about unresolved business challenges.
The panel ultimately agreed that a single definition isn’t available - and might not be helpful for business anyway. Their view reflects the fact that organisations now have access to such vast volumes of data - beyond “big data” alone - that its uses for data science are also broad.
There is an imperative to have a clear understanding of the need to hire a data scientist, and your expectations of the value and outcomes that you want them to deliver, before you set about hiring one. In other words, what can data science do for your business, and how do you create the conditions for them to be successful?
Knowing when to hire a data scientist
With this in mind, the panel explored how organisations can make a business case for appointing a data scientist once they’ve understood how the role can make a difference to operations and outcomes.
The benefits of data science identified by our expert participants include:
Clearly, it’s not a decision to take lightly - and there’s a big risk of jumping on the bandwagon by hiring too soon. Many factors across an entire business must be considered to lay the groundwork that will make the data scientist’s efforts as successful as possible.
Linford stated an existing business intelligence function is vital. An organisation’s raw data should already be available and catalogued, with data governance in place. The data scientist will then be able to understand the data and start to use their “advanced data toolkit” as soon as they’re in post.
Then there’s the perennial issue of relationships between data teams and the IT department. A data scientist who starts in the role without agreement over the necessary data permissions and appropriate controls could have a fight on their hands to achieve their own goals and those of the business. Involving IT during the role scoping process can prevent problems further down the line. For instance, the data scientist might build a product only to find it isn’t compatible with IT’s parameters.
Communication is crucial. There can be a fear of introducing data-led decision-making, agreed the panel, with employees worried that automated processes created by data science will eventually replace them. Sarah said that’s why the whole organisation should be reassured that, in fact, they’ll be able to do higher-value work as a result of data-driven operational improvements.
Finally, in counterpoint to the issues set out above, the panel believed that much smaller businesses that are keen to appoint a data scientist should be bold and hire early. The right expertise can help the firm set up successfully, building the correct business model based on the volume and quality of data available.
Getting the best from your data scientist
Summarising the discussion, the panel returned to their personal experiences of data science as a profession. They cited the components required to ensure the role is as beneficial for the person living and breathing it on a daily basis, as it is for their organisation.
Factors include:
Conclusion: The era of data science
It’s undoubtedly an exciting time to seek a career in data science. The panellists have followed diverse routes into the role, and believe it draws on many skills, subject areas and even personal characteristics.
How has your own blend of these attributes determined which path you’ve taken into a data science role? And how would you describe your own organisation’s approach to data science? We’re keen to hear about your experiences.
IT’S A PEOPLE THING: CREATING A BETTER DATA CULTURE ACROSS GOVERNMENT
DAMA recently hosted a webinar examining the Government’s plans to improve data quality across all departments. The compelling discussion, hosted by our Committee Member Nicola Askham, featured a presentation by James Tucker.
James is Head of the Government Data Quality Hub at the Office for National Statistics. He updated attendees on the Whitehall-led drive to overhaul data quality across the public sector. We also discovered how the role of data professionals can be improved by implementing this future-facing strategy.
You’ll rarely read about seamless government data management projects in the mainstream media. There’s a tendency for journalists to focus on big budgets, difficult deadlines and glitches in delivery which are deemed more interesting for their readers than success stories.
It’s easy to see why this might be deflating for data management professionals working across UK Government departments. However, things should soon start to change for the better thanks to a number of data quality initiatives that are in play.
In 2020, ministers unveiled a new National Data Strategy. This initiative has included the publication of the Data Quality Framework. James says the framework will be used to “get the right data to the right place, at the right time” (to paraphrase the National Audit Office). It will provide a “guiding light for data quality” across departments. These are numerous: 25 ministerial departments, overseeing a total of more than 400 public bodies and agencies, with thousands of employees managing data.
While the framework places focus on providing better data for policymakers it will only succeed with an equal recognition that the people behind the data count, too. That means launching a concerted effort to win their hearts and minds. It’s a case of overhauling outmoded approaches to create “a culture of data quality”.
James leads the Data Quality Hub. Part of the Office for National Statistics, it was established midway through 2021. DAMA has been heavily involved in its launch strategy.
Within the government’s overall data quality drive, building a positive and productive culture is a key part of the Hub’s remit. This will help make the day-to-day experience of data professionals front and centre, rather than departments focusing solely on the data.
It won’t be easy. Departments - and teams within them - are disparate. James says it was difficult to find a succinct definition of culture. So, the Hub decided to explain how culture relates to data quality, and how existing approaches within government impact it, on its own terms.
Data quality management issues identified include: lack of leadership prioritisation; an enduring tolerance to sub-par data quality and a “sticking plaster” mentality; inconsistent approaches and limited knowledge sharing; new data sources bringing unfamiliar challenges; and, perhaps most challenging of all, an inability to determine whether data is fit for purpose.
A new data quality culture will close these gaps, James believes, if it wraps in the following commitments across all departments:
• Shared goals and strategies
• Ongoing data quality assessment
• Creation of best practice internal communities
• Linking departmental approaches to solve emerging challenges
• Closing the gap between data management at source, and data analysis/use
James illustrates the final point by citing an interesting pilot scheme. Data managers working on the emergency services frontline have been trained to understand how analysts might use the information they input, which should in time lead to data-driven policy improvements.
All of these initiatives paint an exciting picture of the future of data management for government services. But as we all know, strategy is one thing; implementation is quite another.
The good news is the Data Quality Framework sets out actionable principles for change. James states these are being adopted to tackle data quality challenges head-on and propel lasting cultural change across government. This will also mark a step-change in how data professionals’ experience their job.
1. A leadership commitment to high-quality data
2. Data quality activity that has users’ needs at the centre
3. Holistic approach to quality assessment, from data collection to use
4. Clear and effective communication of data’s quality in relation to its purpose
5. Better anticipation and understanding of policy/regulatory/technology changes that affect data quality
Ultimately, the principles will underpin a better and more widespread understanding in government that people matter if data quality is to reach new heights. James asserts, however, that the Hub has the right processes in place to guide people on the journey and make a difference.
Steps being taken include: widespread sharing of best practice; training, tools and guidance to build capability; ongoing advice and support, with self-serve products on the way; and a mechanism for data professionals to challenge briefs and help set the direction of their department’s data quality programme.
James says the final point must be prioritised at all levels of seniority, so boosting data literacy is key. However, he adds this will also require everyone who works with data to accept a greater level of accountability.
Conclusion: Quality in, quality out
Sharing the graphic below James concluded by expressing the hope that all data users, across departments, will gravitate to the top-right quadrant:
When that happens, data management for Whitehall departments will truly be a nurturing environment, shifting away from inertia and anxiety towards a culture of learning, and a focus on the future. Innovation and quality will surely follow - and that can only be good news for millions of people who use government services across the UK.
James is keen to learn from external practitioners. To ask him further questions, or to share your own experiences of culture change around data management and data quality at your organisation, please email DQHub@ons.gov.uk
At Data Orchard, we work with organisations - primarily not-for-profits - looking to improve their use of data to make better decisions and achieve greater impact. One of our key offers is assessing data maturity. You can read a bit more about the history of our work with data maturity in our blog, but suffice to say it’s been an ever-deepening fascination since 2010, when ‘big data’ and ‘open data’ really began making waves.
In 2015, after trying and failing to find a framework out ‘in the wild’ that would help give structure to how we think and talk about data with not-for-profits, we took matters into our own hands. In 2017, in partnership with DataKind UK we published our own. To our knowledge, it was the first data maturity framework produced specifically with the not-for-profit sector in mind.
Since then, our theory of data maturity has continued to evolve; we’ve launched our own Data Maturity Assessment Tool to help organisations figure out where they are; and we increasingly work to raise the levels of data maturity in the not-for-profit sector - both as trading consultants and champions of ‘data4good’ in our social cause.
As such, we are often asked to speak about data maturity, data maturity assessments, and what good can come of all this talk about data. In this blog, I wanted to explain a bit about our Data Maturity Assessment Tool and highlight some of the slightly surprising effects that assessing data maturity seems to have on organisations.
Invariably, when we’re asked to talk about data maturity, we find ourselves coming back to the basics… What is data maturity and, while we’re at it… what is data?
We define data maturity as:
‘An organisation’s journey towards improvement and increased capability in using data’
When we talk about data, we have a very broad and holistic definition. We mean all the types of information an organisation collects, stores, analyses, and uses. This could be recorded in many formats: numbers, words, images, video, maps. This means it’s everywhere in an organisation - in every department, in every service and team, and in every job role.
Because data is so pervasive, complex and - yes - messy, it’s really helpful to have a simple framework to give structure to how we think about it.
Our Data Maturity Framework sets out seven key themes and five stages of maturity, from ‘Unaware’, through to ‘Mastering’. The themes are areas that we’ve identified as being crucial when it comes to advancing data maturity. Some are more obvious and practical - like ‘data’ and the ‘tools’ you need to collect, store, and present it. Some are about purpose in how you use it and how you analyse it. And most importantly, three are about people. ‘Leadership’, ‘culture’ and ‘skills’ are essential to making data work for an organisation.
Figure 1 Data Orchard's Data Maturity Framework
To quote Maya Angelou "You can't really know where you're going until you know where you've been". Knowing where you are starting from with data maturity is really important.
After developing the Data Maturity Framework, we immediately found it was great in theory to know what constituted bad, good and great levels of data maturity. But, if it wasn’t easy for organisations (and especially busy leaders with little time or interest in data) to use it to assess where they actually are right now, then it wasn’t helping them plot their path to greatness.
Asking the right questions to assess what stage an organisation is at, under each of the key themes, is the art of the Data Maturity Assessment.
We launched our online tool - the free Data Maturity Assessment Tool - in 2019, and since then have created paid multi-user whole-organisation assessment version. There are also options for other agencies and partners to use our tool with their own clients or members.
The tool is, essentially, a simple online questionnaire that produces a clear, easily digestible report on where an organisation is, based on the answers given. It allows organisations to benchmark themselves against peers, plan their next steps and - depending on engagement across the organisation - identify differing viewpoints across the organisation.
I won’t go into too much detail about the tool itself - you can read about it on our website, and you can even use the quick, free, 5-minute taster version to try it out for yourself (though even the full version only takes 20 minutes to complete).
What I do think it’s interesting to highlight, though, are some of the less obvious benefits that organisations get from going through a data maturity assessment.
Let’s be honest, not everyone loves data the way that we, as data professionals, do. In fact, in our experience, most people actively dislike talking about data. Perhaps more worryingly, our State of the Sector 2020 report found that, in 63% of not-for-profit organisations, the leadership don’t see the value of data.
But, simply by going through the process of completing a data maturity assessment, we find that people from across an organisation (not least leaders) are:
● Encouraged to learn about data, by having to think and talk about it in new ways
● Inspired to come up with new ideas and motivated to get better at data.
I’m not sure if we even appreciated the power of this when we designed the assessment tool, but going through a data maturity assessment is, in many cases, a huge learning opportunity. It increases understanding about data maturity itself, and enables shared thinking and a common language about the challenges an organisation faces.
Yes, it also allows an organisation to measure where they are… But this opportunity to learn, coupled with its effect as a catalyst for further discussion and action, is possibly the biggest benefit we see for many organisations. We’ve lost count of the number of people that have reported back to us that they are suddenly having more positive conversations about data, feeling enthused about making plans, and that they have a better understanding of data and data maturity, simply through taking the assessment.
As ‘data people’, we think anything that gets people nearly as excited as us about talking about data has to be a good thing, and that is borne out in the… erm… data! You can read our 2022 impact report here. One organisation we witnessed become newly enthusiastic about data after conducting a data maturity assessment, is a cancer support charity that completed repeat assessments over the course of three years.
The graph below shows the difference in their data maturity from their baseline assessment in 2018 to their 2021 assessment three years later.
Figure 2 Change in organisation data maturity scores from first assessment in 2018 to fourth assessment in 2021
Clearly, this organisation’s journey didn’t just consist of taking yearly data maturity assessments. Their journey has been supported by various interventions. In the first year Data Orchard provided training and advice, over the next two years there was investment in new tools and training, improving data quality, analysis and reporting. However, that baseline assessment was a key part of engaging and inspiring people in the organisation to take action, and convincing leaders to invest. Repeat assessments have certainly helped sustain the momentum over time, as well as helping them prioritise and plan their next steps….and they’re looking for analytical skills on their leadership team too!
If you’re interested in watching my DAMA UK webinar on Advancing Organisation’s Data Maturity, it’s available here.
If you’re interested in finding out more about Data Orchard’s work in data maturity, or think you or your clients could benefit from a Data Maturity Assessment, we’d love for you to get in touch.
We also have a small but growing group of data professionals in our Slack group, aimed at connecting a community of data people who can share knowledge and experiences. Why not join?
Aaron Bradshaw, Data Governance & Enablement Specilst, Alation.
This is a follow on blog from the DAMA UK session (Data Ethics as Business Opportunity) held on 12:00 GMT 1st July 2022, hosted on BrightTalk.
Light is both a wave and a particle. Data ethics is both an imperative and an opportunity. New regulations covering data privacy and other ethical concerns require that enterprises govern internal data processes according to these new laws. And, while change at large organisations is tough, data leaders would be wise to reframe such transformations as business opportunities rather than burdens.
I recently led an online session, Data Monetisation and Governance, looking at the evolution of data governance, defining data ethics (from the Turing Institute), and touching on the balancing act between using data to monetise (by increasing revenue, decreasing spend, or mitigating risk) and meeting ethical obligations. In other words, ethics and governance aren’t just about mitigating risk; with the right approach, they can boost profits, productivity, and ROI.
The session began with an audience poll. I asked attendees:
● How often do you think about data ethics?
● What does data ethics mean to you?
Poll of attendees revealing the data ethics is low on their priority list
Interestingly, from a pool of data professionals, the vast majority think about data ethics just a few times a month. Contrast this response to a wider audience of non-data practitioners and this response changes to rarely/never.
Why is data ethics overlooked? When it works well, it doesn’t make headlines. I raised the Cambridge Analytica Scandal and pointed out how it is often only when these stories hit the news that people question the ethics behind how companies are using data.
In the Cambridge Analytica case, the company went from a data strategy focused on monetisation by increased revenue to company closure due to the reputational damage from the negative media and public response. Clearly, using private Facebook data collected in a nefarious manner to sway political elections is not ethical. In the court of public opinion, Cambridge Analytica had violated a clear ethical boundary.
Should individuals have autonomy over their personal digital data? The growing consensus is that individuals should have a say in how their private data is collected and used. People are demanding they have the choice to opt out of personal data sharing. Multiple regulations across the globe (GDPR, CCPA, CPRA, POPIA, HIPAA, PIPEDA, LGPD) are rising to this demand. Such laws are pushing the rights of the individual, ultimately trying to give everyone their own decision-making ability around how their private data is collected and used.
This presents a challenge to data practitioners, and an opportunity. Meeting regulations such as GDPR takes a huge amount of effort. A narrow focus has meant that a lot of organisations haven’t taken a step back and frankly assessed the collateral created to transform the organisation. Yet this collateral is valuable for more than just meeting GDPR demands! Organisations who do assess can gain additional benefits from the work that’s already done.
Indeed, ethical data practices can actually support data monetisation strategies. In the session, I walked through the matrix below, sharing examples of how organisations have used ethical standards to monetise or where monetisation has driven ethical innovation.
What’s your data strategy? Offensive strategies position data as product, while defensive look to data as insight. All strategy types offer opportunities for the business.
Under GDPR Article 30, the Record of Processing Activity (ROPA) document needs to be created and maintained. This effectively collates every process in an organisation that uses personal data, the type of personal data, what the process does, etc.
In an organisation with 500 processes, there is overhead to maintain the ROPA. Each process has a cost and a value (for example, the FTE cost divided by the time spent annually, plus infrastructure cost, against attributed revenue from the process). Once the ROPA has been created, organisations can review all operations from a bird’s-eye view, identify costly processes that are no longer effective, and decommission them.
● Decreased spend: One business saved £100,000 on average per process decommissioned and enjoyed less ROPA maintenance.
● Reduced risk: Less processing of personal data, lessened chance of breach, saving up to 4% of global turnover for GDPR fine mitigation.
● Increased efficiency: FTE resources can be reallocated to rewarding tasks that add value.
This was a great example of how ethical standards mandated in a personal data privacy regulation can be used to create value for a business.
Financial services businesses must meet extensive regulatory requirements, which demand full governance including: Data ownership, definitions, agreed-upon data quality rules and results, and lineage (BCBS239, CCAR, etc.).
Data governance exposes inefficiency. Many processes executed in silos for decades require numerous manual steps. However, these manual steps weren’t transparent until active data governance required it. Delivery of governance around these processes can unveil massive, inefficient processes with multiple manual (and often redundant) steps.
In my time working as part of the data teams in multiple financial services organisations, I’ve seen companies revisit processes due to data governance. With governance-as-guide, those organisations can simplify onerous processes, reducing 25+ stages with 10+ manual steps to just 15 stages with 5 manual steps, for example.
● Reduced cost: Finding and eliminating wasteful steps and processes saves time and money.
● Reduced risk: Streamlined processes reduce the chances of data being misused or untraceable.
● Digital transformation: Moving from end-user computing (and the associated benefits of disaster recovery, access control, and automated data quality), as well as faster processing times and improved operational efficiency.
The 2007/2008 financial crisis unveiled the monstrous risk of mis-reporting data. In its wake, many data leaders have made ethical standards core to their operations. This focus has led to simplified legacy processes, reduced total steps, and minimised manual effort (all of which contribute to lower costs and improved efficiencies!).
At a credit card company, there was an initiative to work with mobile phone networks to share data.
Imagine the scenario: You’re going on holiday in a foreign country. You land and disembark the plane. In the terminal you switch your phone on and, within a few minutes, you get a message from your credit card provider. It knows you’re overseas, offers you the chance to disable overseas spending, or extends a personalized 0% foreign exchange fee just for you.
For some people, these are received as great benefits that make their lives easier. However, for others, this can lead to impulse spending and they may not want to receive these. Further still, some people will not like the thought of these data exchanges occurring between companies they patronize.
Nicola Askham raised the concept of a “Daily Mail Test”. If the media could make an embarrassing headline out of your data usage, then it’s probably out of most people's ethical limits.
Why do we collect data? What is our duty to the individuals whose data we’ve captured? What does it mean to use this data ethically?
Such questions capture the complexities of data ethics today and reveal why some argue that data philosophers will be the new data scientists. Ultimately, every individual will have differing thoughts on what appropriate data usage means. Regulations will eventually empower people to exercise control over how organizations manage their personal data.
For data leaders facing such laws, communication around these topics is vital. The important thing is that, as a collective of data professionals, we need to promote and increase the conversation around these data uses, whether that is increased discussion at Data Councils to gain a consensus of acceptable uses within an organisation or raising awareness across the various users looking to gain insight and value from data.
CTA: See how ethics can support ROI. Watch the full presentation here.
