Recent posts

Categories 

<< First  < Prev   1   2   3   4   5   ...   Next >  Last >> 
  • 5 Feb 2025 15:37 | Sue Russell (Administrator)

    Data Quality and AI/ML – The Best of Friends?  Nigel Turner, Principal Consultant, EMEA

    Artificial Intelligence (AI) & Machine Learning (ML) have become the focus of global news headlines over the past months.  Pessimists fear that the growth of AI/ ML poses a serious threat to the future of humanity, invoking Terminator style doomsday scenarios. Optimists claim AI/ML can be the saviour of humankind, a vital tool in helping us identify and avoid impending future problems and disasters, often before we are aware of them. 

    The reality is, as always, somewhere between the two.  Like any new set of technologies AI/ML has the potential to benefit us all if applied ethically and intelligently.  A growing library of use cases is already beginning to appear which show how AI/ML can identify and help to create new opportunities and resolve problems in areas such as government, retail, banking, insurance, manufacturing, travel etc.  But if used wrongly or for morally questionable purposes such as the dissemination of political misinformation AI/ML could cause intended or unintended harm.  So what can we do as data management specialists to play our part to ensure that AI/ML is a force for good, and not a force for evil?

    One key way is to recognise and promote the fact that AI/ML, like any set of technologies that relies on data is only as good as the data it is given to work with.  However carefully the algorithms that drive AI/ML are constructed and applied, they will invariably produce false outcomes if the source data is not a true reflection of the reality that data is supposed to represent.  Put simply, AI/ML critically relies on good data quality.  Feeding AI/ML with inaccurate and incomplete data inevitably results in it generating outcomes, decisions and actions that are inaccurate, unreliable, misleading and potentially downright dangerous.

    To use a simple analogous example, AI/ML could be tasked with solving a jigsaw puzzle. It would have to be taught the rules of how a jigsaw works, in particular recognising that a complete picture has to be built from the 1,000 jigsaw pieces it is presented with.  As we would do, it could start by identifying the four corner pieces and remaining frame pieces identified by having one straight edge. It could then assemble the pieces to form the complete frame, and progress its completion from there.   If all 1,000 pieces were present and correct, this is an achievable task.  But what if some jigsaw pieces were missing, some pieces were duplicates of other pieces, and other pieces were from a totally different jigsaw?  Suddenly the task becomes much harder and the outcome less certain and reliable.  A similar process also applies to data.  If pieces of required data are missing, duplicated or invalid, AI/ML may struggle to create the finished picture intended.  Worse still, it could generate a different picture altogether. 

    So getting data quality right is a ‘must have’ for effective AI/ML.  Yet this is not the reality for many organisations who are using or thinking of applying AI/ML.  A recent Cap Gemini survey found that 72% of business and technology executives stated that the biggest barrier to implementing AI/ML and data analytics in their businesses was fragmented and poor quality data. [1]  And it must be stressed that this is not just a problem for AI/ML or data analytics.  Poor data quality continues to hurt business profitability, efficiency, productivity and decision making.  A 2023 survey by Drexel University & Precisely found that poor data quality is ‘pervasive’ in most organisations with 66% of respondents rating the quality of their data as ‘average, low or very low’ [2].  So how do you spot poor quality data? Its main symptoms are:

    • Missing data, where fields are blank when they should contain relevant information, e.g. a date of birth. 
    • Inaccurate data, where data stored and processed does not reflect the real world, for instance an incorrect or invalid product number.
    • Duplicated data, where multiple variations of the same data exist in a data source, e.g. the same customer inadvertently appearing many times as different records in a CRM database
    • Inconsistent data, where data that should be consistent across data sources varies, for instance different country code tables used as reference data in various applications which are supposed to identify the same country, so different codes may identify the same country, or vice versa.  This is usually indicative of the lack of agreed data standards.

    Given that these data quality problems are sadly pervasive across the great majority of companies, some have argued that investing in AI/ML is pointless and a waste of money until and if these data quality issues are first identified and resolved.  But this is wrong, and misunderstands the nature of data quality and how to tackle it.  The world changes constantly and so maintaining good data quality is a continuous process, not a series of one-off data cleansing challenges.  The logic of this would mean that AI/ML will never be deployed and its potential benefits never realised. One simple and obvious way to resolve it is to make data quality improvement an integral and essential stage of any AI/ML project, with a critical early step being to analyse any proposed AI/ML data sources, and identify and resolve any important data quality problems found before AI/ML is applied.  The chances of a successful AI/ML project are then greatly increased by doing this.        

    Moreover the relationship between AI/ML and data quality is not a one way street.  Whereas effective AI/ML depends on good data quality, AI/ML can itself be used to help to solve data quality problems.   Many data quality software vendors have recognised this and are already embedding AI/ML functionality into their toolsets.  AI/ML can help address existing data quality problems and proactively prevent future problems by:

    • Automating data capture, so there is less reliance on manual input and the errors that inevitably result from human error.
    • Validating data entry, so that attempts to input data that does not meet preset data quality standards are rejected.
    • Discovering and enforcing data quality rules.  Through its inbuilt learning capabilities AI/ML can derive its own rules that it applies to data and so can identify and reject outliers and data anomalies. 
    • Identifying duplicate records.  Again AI/ML can be used to analyse a data source and identify unintentional duplicate records, and potentially match and merge them.
    • Filling missing data where AI/ML can deduce what the gaps should contain and complete them, potentially by accessing third party data sources.  

    Poor data quality is indeed an enemy of AI/ML, but using AI/ML approaches and capabilities to identify and tackle data quality problems is a clear win-win.  Better data quality will make AI/ML more effective and useful; AI/ML can help to create the better data it needs to improve its business value.  Using the techniques of data quality and AI/ML in tandem can bring mutual benefit and better business outcomes.  Whereas today AI/ML and data quality can often be presented as enemies, they can and should become the best of friends.  

    [1] Cited in “Intelligent MDM, 2nd Informatica Special Edition”, Lawrence C. Miller, 2023

    [2] “2023 Data Integrity Trends and Insights Report”, LeBow College of Business, Brexel University and Precisely, 2023

  • 5 Feb 2025 15:31 | Sue Russell (Administrator)

    November 2024, Nigel Turner, Principal Consultant, EMEA

    Earlier this year I was asked to take part in a panel session at an event in Manchester, UK. The main focus of the session was Artificial Intelligence (AI), this year’s hot subject in data management, and probably a focus of headlines and scrutiny for some time to come. Much of the panel discussion focused on how AI should be governed, both by the organizations and individuals who develop and exploit it, and by governments who seek to regulate it so it benefits, and does not detriment, society as a whole. The interrelationship between AI and data was a central discussion point, and this led to consideration of how the governance of AI and the governance of data should relate to each other. 

    One question centred on whether AI governance and data governance were totally different disciplines or had similarities and so overlapped in any way. At the outset there was a broad consensus that both data and AI needed stronger governance than is often the case. Lately AI has been hitting the headlines for the wrong reasons in the UK, with several reported cases of the technology being used to simulate fake statements from well known politicians and celebrities. These stories exemplify that doubts are growing about AI’s ability to differentiate fact from fiction, leading to it being too readily able to distort and create a false reality. Moreover, combining these emerging capabilities with longstanding data quality and other problems can lead to undesirable conclusions and outcomes. It's therefore evident that both AI and data need careful governance, with policies, processes and procedures put in place to minimise the chances of mistakes and errors. 

    So how do you ensure AI and data are governed effectively to protect individuals, organisations and society from harm? First, it’s clear that for both AI and data you need rigorously defined, implemented and enforced governance frameworks, set within a regulatory and operational context. As a more established discipline, data governance has relatively mature frameworks developed over several years. These lay out how organisations should ensure that their employees and agents are accountable for data, personally responsible for its use and curation within legal and ethical boundaries. A good example is the General Data Protection Regulation (GDPR) in Europe, a comprehensive legal data protection framework which is being emulated in many other parts of the world. Problems with data often arise when organisations do not implement effective data governance frameworks and controls, and as a result inadvertently allow their data to be used in uncontrolled and often unethical ways. Applying AI technologies on top of this is a recipe for disaster.

    Although legal and operational AI frameworks are also beginning to be being developed, they remain immature. This is not a new problem in data management as it is often the case that new technology outpaces the ability of society, lawmakers and organisations to control its use effectively. And again, with AI, effective governance is playing a catch up game, trying to keep pace with the technology’s rapid and accelerating capabilities.  The good news is that many governments and organisations have at least recognised the need to act. The European Union, UK, USA, Canada, Brazil and China are all drafting AI legislation and associated frameworks and guidelines. These emerging legislative frameworks cover issues such as determining when the use of AI is appropriate (or not), what ‘guardrails’ need to be put in place to limit what the tool is and is not allowed to do, and emphasise key safety, privacy and ethical policies, principles and practices that must be adhered to. They are also stipulating how AI outputs should be validated to avoid bias and ensuring that AI is designed and implemented transparently so its results can be validated or critiqued by others.

    But the approaches vary significantly. The UK’s current AI governance emphasis is to create a ‘principles based framework’ for AI, rather than enacting detailed legal regulations on how AI should be managed and controlled. The five principles focus on:

    • AI systems must operate in a robust, secure and safe way
    • They should be transparent and explainable
    • They should be fair, and not undermine the legal rights of individuals or organizations
    • AI governance should be put in place to ensure effective oversight of the AI lifecycle and that accountability is clearly defined
    • Impacted individuals or organizations have the right to challenge an AI generated decision or outcome. 

    The monitoring of the above will be sector based, using existing industry and data protection regulators to scrutinise compliance with these principles. Whether this will work or not remains to be seen. It’s also noticeable that these principles apply equally to data as to AI, as either or both can breach these principles. So, it’s vitally important that AI governance and data governance must align and reinforce each other. Both data inaccuracies and AI imperfections lead to undesirable outcomes. 

    The fundamental relationship between AI and data governance is that, as we all know, AI’s results rely heavily on the quality and veracity of the data it is trained on. Poor data will produce erroneous outcomes and conclusions; good data is likely to generate valid and useful insights. So, any organisation considering using AI operationally needs to ensure that not only does it have a robust AI framework in place, but also an equally well implemented data governance framework. Data governance must underpin and supplement AI and provide the sound and reliable data management foundation that AI ultimately depends upon.

    Data governance and its drive to create and maintain high quality, trusted data is a necessary precondition of AI success. But AI can also enable successful data governance and is already beginning to do so. One of the big challenges data governance continues to face is how to leverage overall control over what are often siloed, disparate, inconsistent and disconnected data sources. AI can help to break down these silos by helping to identify, categorise and consolidate related data found in these stovepipes, for example highlighting personal data held in various scattered source files. This enables data governance roles such as data owners and data stewards to get a better handle on this data and so improve its overall use and control.  Data governance also critically relies on the effective generation and management of metadata which enables data owners and data stewards to curate the data for which they are responsible. Many data catalogs are already embracing AI technologies to help automate the generation of the metadata required and to dynamically update it as source data changes. Furthermore, AI can also help to suggest, automate and enforce data governance policies and rules, including data quality rules. 

    To summarize, AI governance and data governance are both ‘must haves’ for any organization which wants to exploit its data assets within legal and ethical boundaries. In order to ensure that they work to reinforce and not negate or contradict each other AI and data governance specialists within organisations need to get together to develop and implement the required frameworks harmoniously. Only this will provide a seamless and integrated set of frameworks where the benefits of AI can be realised more effectively within the controlled and secure data environment that data governance can bring. This will require AI and data governance people to deepen their understanding of each other’s principles, policies, approaches and tools. If this is done, both AI and data governance are enhanced, and the disasters highlighted above reduced. 

    A win-win scenario is achievable but will take motivation and effort from the data management community. If this comes to pass, the opportunities for exploiting an organization’s data assets in an ethical and managed way are potentially boundless.  

  • 2 Feb 2025 12:03 | Sue Russell (Administrator)

    Every year, the DAMA UK committee gathers to reflect on our progress in supporting data professionals across the UK. As a volunteer-led organisation, our committee members dedicate their time and expertise to advancing the field of data management. This year, we were delighted to welcome Antony Marlow to the committee, alongside our existing members elected last year. 

    As many of you have heard me say before, the data profession is still in its early stages compared to well-established fields such as accountancy and medicine. With the rapid advancements in artificial intelligence, it has never been more crucial for DAMA UK to collaborate with data professionals both in the UK and globally to drive higher standards and recognition for our industry. 

    Here’s an overview of our discussions and progress against our key objectives: 

    1. Expanding Our Membership 

    Objective: Grow our dynamic community and welcome more professionals into DAMA UK. 

    Progress: Corporate membership has seen significant growth in recent years, highlighting the increasing recognition of data management’s importance to business success. 

    Next Steps: We are committed to further enhancing the value we offer to corporate members and exploring additional ways to support them. 

    2. Raising Industry Standards 

    Objective: Promote the professionalisation of data management by increasing the number of certified data professionals and advocating for their recognition on par with other professions. 

    Progress: 

    A growing number of professionals are obtaining their Certified Data Management Professional (CDMP) qualification and joining the global community of certified practitioners. 

     

    Our mentoring programme is pairing more experienced data professionals than ever before with mentees.  

    Next Steps: We aim to support professionals beyond qualification. Our Training sub-committee is actively reviewing how we can enhance our training offerings, both for those seeking certification and those who have already qualified. 

    3. Enhancing Member Support 

    Objective: Provide our members with the best possible guidance, resources, and networking opportunities. 

    Progress: 

    • Over the past year, we have hosted 15 webinars, with 3,751 data professionals registering, watching, or attending live sessions. Our most popular webinar to date focused on Data Literacy 101. 

    • Engagement on our LinkedIn pages continues to grow, with increasing interaction from our community. 

    • We have launched regional meet-ups, including in Scotland and the North, to facilitate local networking opportunities. 

    Next Steps: 

    • We have an exciting lineup of webinars planned, but we always welcome new topics and speakers. If you’d like to host a session, let us know! 

    • We will continue to expand our regional events to better serve our members across different geographical areas. 

    4. Regularly Reviewing Our Offerings 

    Objective: Ensure our resources, training, and information remain relevant in an ever-evolving data landscape. 

    Progress: Updates to our website are underway to improve accessibility and usability for our members. 

    Next Steps: AI is an essential topic in today’s data conversations, and we are exploring ways to incorporate AI-related content into our offerings. Stay tuned for the launch of our refreshed website later this year! 

    5. Strengthening International Collaboration 

    Objective: Foster partnerships with DAMA chapters worldwide to share best practices and drive the profession forward. 

    Progress: We are actively collaborating with international DAMA chapters, exchanging insights and supporting new chapters as they emerge. 

    Next Steps: We will continue to strengthen these global connections. Additionally, DAMA EMEA has recently published an International Data Maturity Benchmark Report (January 2025), which we look forward to sharing this with our members. 

    Looking Ahead 

    As one of the largest DAMA chapters globally, we recognise that there is always more to do. With technology evolving at an unprecedented pace, our work has never been more important. It is a privilege to collaborate with such a dedicated group of volunteers on the DAMA UK committee, and we are always eager to hear from our members. As a membership organisation, we exist to serve our community—so please reach out with your thoughts, ideas, and feedback. 

    If you’re not yet a member, now is the perfect time to join! 

    What’s Coming Up? 

    The next 12 months promise to be eventful, with several key milestones on the horizon: 

    • Spring: Website upgrade and member survey 

    • Summer: DAMA UK elections (held every two years to elect committee members) 

    • Autumn: DAMA UK Awards 

    We look forward to another successful year of supporting data professionals and driving positive change in the industry! 

  • 4 Dec 2024 08:03 | Sue Russell (Administrator)

    Today I’ve been reading the ONS ‘Lessons Learnt Review’ regarding the delayed project to overhaul the Labour Force Survey. The short version: systematic underinvestment, and recurring overoptimism… an experience the data management community knows all too well.

    We each see it in our own organisations, with leadership waxing lyrical about being a data- or AI-enabled business, but rarely giving data management the budget to deliver. Every day, DAMA UK members work wonders with the resources they’re given, while striving to contain their execs’ wilder ambitions, and advocate for proportionate investment. It will come as no surprise that politicians need the same kind of reality-checks.

    That’s why, last week, DAMA UK compiled and submitted a response to the UK government’s industrial strategy consultation. [link to response]

    The strategy that follows from this exercise will have implications for all of us: as employees, business owners, citizens, consumers, and users of public services. As a strategy with data at its core, our profession cannot leave it to non-experts! If we want an industrial strategy as robust and effective as it can be, we going to have to give of our expertise to point it in the right direction.

    It’s important to note that while this is certainly political engagement, DAMA UK engages from a position of political independence. Our members are a diverse bunch, covering different regions, demographics, industries and sectors, and different political tendencies. That’s our strength. It allows us to give a balanced view, free of the constraints of narrow experience or allegiances, and unified by our professional expertise.

    In assembling our response to the industrial strategy consultation, I was struck both by our members’ breadth of knowledge, and by the real unity on many points. One recurring theme was a real welcome for the centrality of data in the government’s proposals, and the priority given to sharing of national data assets. On the flip side, members consistently and constructively called for realism, and serious investment in data management capabilities, especially in the public sector, to match the strategic ambitions.

    Our submission says, loud and clear, “Data is worth doing, so it’s worth doing well.”

    Interestingly, workforce data such as the Labour Force Survey got a specific mention, as one of the national data assets most critical to the delivery of industrial strategy. The ONS review gives a timely reminder to the teams developing the strategy, that effective data can’t be wished into being. Data management experts throughout the public sector and the wider economy, must be supported, listened to, and properly resourced to play their starring role in delivering this strategy.

    DAMA UK’s mission is to nurture a community of data professionals who champion data management. Our submission to the consultation hits every note. Each of you who contributed, whether by survey, email, webinar (or via my LinkedIn DMs!) is part of a community action. We are now on record, as experts, advocating for our discipline, and offering professional insight to improve an important national policy.

    Let’s do more of this!

    • I would love to hear your thoughts on the final submission: what would you add, and what would you challenge?
    • Do you know of other consultations – UK-wide or devolved administrations – that DAMA UK should have a say in?
    • How can we best engage members to develop DAMA UK policy positions in future?        
    Read the DAMA UK Response - DamaUK_ResponseSummary.pdf
        
  • 6 Dec 2023 15:54 | Sue Russell (Administrator)

    I joined DAMA UK to connect and exchange ideas with other data specialists. The DAMA EMEA (Europe, Middle East and Africa) Chapters conference was an opportunity to turbo-charge that goal. When some of my committee colleagues suggested I go to present the findings of the UK chapter’s survey, I jumped at the chance.

    This year’s meeting was a hybrid event, hosted in Bologna, with about 1000 virtual and 200 in-person participants, representing 60 countries, hundreds of organisations, and all kinds of data, technology and business functions.

    The first day covered a wealth of big ideas in data practice. They were organised into three parallel streams, and the only grumble I heard was that everybody wanted to go to everything at the same time!

    I started with a panel discussion featuring experts in the health sector, each effecting deep, and genuinely inspiring, transformation at very different stages of their organisational lifecycles.

    I dropped into a great talk on socialising AI into our organisations: how to engage business-experts, especially the decision-makers, to frame the questions; and how to find ‘friendly’ ways to get them hands-on with the technology itself.

    I went to see my friend, Ole, who literally wrote the book on Data Catalogues, walk us through the synergy with knowledge graph (spoiler: it doesn’t get you off the hook of data management; you have to organise your data first to unlock the power of any catalogue).

    Then another panel discussion, looking at the democratisation of environmental data. Then another presentation, on what to expect from the impending EU AI Act.

    By the end of the day I needed a lie down in a darkened room just to digest the dozens of new ideas (and quite a lot of tiramisu) I’d already consumed.

    Day 2 brought the focus, largely, onto the work of the DAMA chapters themselves.

    We heard from groups translating the DMBoK into their native languages, and the significance of that work in growing awareness of data management beyond the professional community. My mind was blown by hearing that the Finnish language has a single word meaning ‘data’ AND ‘information’ AND ‘knowledge’; that’s a challenge!

    We had an exhilarating presentation from our youngest chapter, DAMA Egypt, who have really hit the ground at a sprint. It’s just two months since they were officially recognised by DAMA International, and already they have set up study groups, schools outreach, and are engaging with regulators to offer expert input into Egypt’s singular data protection laws.

    I’ll admit, I was proud to hear my committee colleagues, Nigel Turner and Mark Humphries talk us through how DAMA UK has grown into far-and-away the biggest chapter in the EMEA region, and how we are having to reorganise our operations and provision to keep pace with our growing membership.

    I spent the first half of my afternoon in a fantastic workshop, exploring the implications for data management of AI law. While it focused on the EU AI Act, this exercise gave a structure to help unpack data management’s role in the ethics and effectiveness of AI, as well as in the international race to regulate. If anyone would be interested in a DAMA UK version, I would love to hear from you!

    Finally, I dropped back into the DAMA coordination meeting in time for a session on defining our shared objectives for the coming year. The UK delegates split our focus across themes: I chipped in on curriculum development and engagement with education; Mark on an ambitious vision of data maturity assessment at national scales; and Nigel on plans for the next conference…

    Yes, there will be a next DAMA EMEA Conference, although that’s really as much detail as we have for now. It might be in 2024, or maybe 2025. We don’t know where, or by which chapter it will be hosted; possibly France or possibly even the UK. What I can say with confidence is that it will be worth going to.

    Most strikingly, in its content and atmosphere, this was a community event: very different from the huge, commercially-oriented data conference we see filling our LinkedIn feeds and London’s exhibition centres.


    Everyone, including the corporate sponsors, came willing to participate as equals. I arrived with just a handful of social media contacts, and left having established some wonderful friendships across continents!

    As I said in my talk - more than anything, in our mission statement, and even on our logo - we are a community, working together and learning from each other to promote the value of our profession. The rewards are best – the inspiration and the empowerment – when we engage as a community: by showing up and chipping in at webinars; or making the effort to get to a regional meet-up; or trekking halfway round Italy to be inundated with learning and desserts.

    As we roll into a new year, I really encourage you to think how you might engage a little bit more, and get a lot more out of your DAMA UK membership in 2024.

  • 30 Oct 2023 11:44 | Sue Russell (Administrator)

    In a world increasingly reliant on data, access to high-quality data and information it is crucial for driving innovation, economic growth, and societal development. The Department for Science Innovation and Technology (DSIT) is carrying out a review of what public sector data has the potential to unlock most value to both industry and the wider economy. Put in simple terms, is there more data government could share and if so, what data is it?  

    DAMA UK joined forces worked with DSIT canvassing our DAMA UK members. We held two insightful roundtable discussions. These sessions aimed to explore views from our members on ways of increasing access to public sector data. This builds on our members previous comments on the government’s National Data Strategy. DSIT are also working with other organisations and carrying out interviews with experts from many sectors to inform this review.  

    Why do the review now?  

    The Pro-innovation Regulation of Technologies Review: Digital Technologiesfound thatgovernment and broader public sector bodies hold significant data but that the ability of the private sector to access this data is inconsistent and fragmented. 

    DSIT is working with Centre for Digital and Data Office (CDDO). You may have seen the work from the CDDO on the data market place. This is looking to share government data across other government departments. It is looking at Essential Shared Data Assets (ESDA’s). Allowing government to share the most important data across departments. By DSIT and CDDO working together, it covers both government and private sector requirements for data.  

    What did our members say? 

    Our discussions covered the current public sector data sharing ecosystem. With our members from various sectors sharing their experiences, providing valuable insights into the state of data sharing in the public sector 

    Value was a key question, and value for whom? Understanding the stakeholders and beneficiaries of data is fundamental to harnessing its potential. This value is more than monetary, it can be to society or to the environment. The value of geospatial data, particularly core reference data like addresses, is indisputable. However, its availability for free to the public sector and must be paid for by everyone else, can create power imbalances, limiting its potential for businesses and for innovation. This issue aligns with the European Open Data Directive, which promotes open access to valuable datasets which are largely geospatial and core reference data.  

    We covered whetherFreedom of Information requests (FOI) hold insights into the data people required from government. And whether Large Language Models (LLMs) creating shadow data, could potentially eliminate the need for official government data sources. We decided shadow data was good for some things, but there would always be a need for government data.  

    Enhancing the discoverability of data was another challenge. Some DAMA UK members had not come across data.gov.uk or the Defra Data Services Platform that hold a wealth of data held within government 

    All in all, it was excellent for DAMA UK member to contribute to this review, and we look forward to the outcome of the work in the coming months.
  • 25 Sep 2023 13:49 | Sue Russell (Administrator)

    Artificial Intelligence (AI) & Machine Learning (ML) have become the focus of global news headlines over the past months.  Pessimists fear that the growth of AI/ ML poses a serious threat to the future of humanity, invoking Terminator style doomsday scenarios. Optimists claim AI/ML can be the saviour of humankind, a vital tool in helping us identify and avoid impending future problems and disasters, often before we are aware of them. 

    The reality is, as always, somewhere between the two.  Like any new set of technologies AI/ML has the potential to benefit us all if applied ethically and intelligently.  A growing library of use cases is already beginning to appear which show how AI/ML can identify and help to create new opportunities and resolve problems in areas such as government, retail, banking, insurance, manufacturing, travel etc.  But if used wrongly or for morally questionable purposes such as the dissemination of political misinformation AI/ML could cause intended or unintended harm.  So what can we do as data management specialists to play our part to ensure that AI/ML is a force for good, and not a force for evil?

    One key way is to recognise and promote the fact that AI/ML, like any set of technologies that relies on data is only as good as the data it is given to work with.  However carefully the algorithms that drive AI/ML are constructed and applied, they will invariably produce false outcomes if the source data is not a true reflection of the reality that data is supposed to represent.  Put simply, AI/ML critically relies on good data quality.  Feeding AI/ML with inaccurate and incomplete data inevitably results in it generating outcomes, decisions and actions that are inaccurate, unreliable, misleading and potentially downright dangerous.

    To use a simple analogous example, AI/ML could be tasked with solving a jigsaw puzzle. It would have to be taught the rules of how a jigsaw works, in particular recognising that a complete picture has to be built from the 1,000 jigsaw pieces it is presented with.  As we would do, it could start by identifying the four corner pieces and remaining frame pieces identified by having one straight edge. It could then assemble the pieces to form the complete frame, and progress its completion from there.   If all 1,000 pieces were present and correct, this is an achievable task.  But what if some jigsaw pieces were missing, some pieces were duplicates of other pieces, and other pieces were from a totally different jigsaw?  Suddenly the task becomes much harder and the outcome less certain and reliable.  A similar process also applies to data.  If pieces of required data are missing, duplicated or invalid, AI/ML may struggle to create the finished picture intended.  Worse still, it could generate a different picture altogether. 

    So getting data quality right is a ‘must have’ for effective AI/ML.  Yet this is not the reality for many organisations who are using or thinking of applying AI/ML.  A recent Capgemini survey found that 72% of business and technology executives stated that the biggest barrier to implementing AI/ML and data analytics in their businesses was fragmented and poor quality data. [1]  And it must be stressed that this is not just a problem for AI/ML or data analytics.  Poor data quality continues to hurt business profitability, efficiency, productivity and decision making.  A 2023 survey by Drexel University & Precisely found that poor data quality is ‘pervasive’ in most organisations with 66% of respondents rating the quality of their data as ‘average, low or very low’ [2].  So how do you spot poor quality data? Its main symptoms are:

    • Missing data, where fields are blank when they should contain relevant information, e.g. a date of birth. 
    • Inaccurate data, where data stored and processed does not reflect the real world, for instance an incorrect or invalid product number.
    • Duplicated data, where multiple variations of the same data exist in a data source, e.g. the same customer inadvertently appearing many times as different records in a CRM database.
    • Inconsistent data, where data that should be consistent across data sources varies, for instance different country code tables used as reference data in various applications which are supposed to identify the same country, so different codes may identify the same country, or vice versa.  This is usually indicative of the lack of agreed data standards.
    • Given that these data quality problems are sadly pervasive across the great majority of companies, some have argued that investing in AI/ML is pointless and a waste of money until and if these data quality issues are first identified and resolved.  But this is wrong, and misunderstands the nature of data quality and how to tackle it.  The world changes constantly and so maintaining good data quality is a continuous process, not a series of one-off data cleansing challenges.  The logic of this would mean that AI/ML will never be deployed and its potential benefits never realised. One simple and obvious way to resolve it is to make data quality improvement an integral and essential stage of any AI/ML project, with a critical early step being to analyse any proposed AI/ML data sources, and identify and resolve any important data quality problems found before AI/ML is applied.  The chances of a successful AI/ML project are then greatly increased by doing this.        

    Moreover the relationship between AI/ML and data quality is not a one way street.  Whereas effective AI/ML depends on good data quality, AI/ML can itself be used to help to solve data quality problems.   Many data quality software vendors have recognised this and are already embedding AI/ML functionality into their toolsets.  AI/ML can help address existing data quality problems and proactively prevent future problems by:

    • Automating data capture, so there is less reliance on manual input and the errors that inevitably result from human error.
    • Validating data entry, so that attempts to input data that does not meet preset data quality standards are rejected.
    • Discovering and enforcing data quality rules.  Through its inbuilt learning capabilities AI/ML can derive its own rules that it applies to data and so can identify and reject outliers and data anomalies. 
    • Identifying duplicate records.  Again AI/ML can be used to analyse a data source and identify unintentional duplicate records, and potentially match and merge them.
    • Filling missing data where AI/ML can deduce what the gaps should contain and complete them, potentially by accessing third party data sources.   

    Poor data quality is indeed an enemy of AI/ML, but using AI/ML approaches and capabilities to identify and tackle data quality problems is a clear win-win.  Better data quality will make AI/ML more effective and useful; AI/ML can help to create the better data it needs to improve its business value.  Using the techniques of data quality and AI/ML in tandem can bring mutual benefit and better business outcomes.  Whereas today AI/ML and data quality can often be presented as enemies, they can and should become the best of friends.  

    [1] Cited in “Intelligent MDM, 2nd Informatica Special Edition”, Lawrence C. Miller, 2023

    [2] “2023 Data Integrity Trends and Insights Report”, LeBow College of Business, Brexel University and Precisely, 2023

  • 25 Sep 2023 10:11 | Sue Russell (Administrator)

    Embracing Change: A New Era for DAMA UK and the Data Profession in the UK

    I’m Lisa Allen, I have the immense pleasure to take on the role of Chair of DAMA UK and I would like to welcome our new committee and extend my heartfelt thanks to the outgoing members. As the Chair of DAMA UK, I am excited to embark on this journey with fresh faces and experienced hands alike, because the next two years are poised to be pivotal for DAMA UK and for the data profession.

    First and foremost, I want to express my deep appreciation for the dedication and hard work of our outgoing committee members. Your tireless efforts saw DAMA UK's growth and success. Your commitment to advancing the data profession has not gone unnoticed, and we owe you a debt of gratitude for your invaluable contributions so a heartfelt thanks to Nicola Askham, Ian Chapman, Aktar Ali, Andrew Lunt, Phil Jones and James Tucker.

    As we bid farewell to some, we also warmly welcome the new committee. This year we saw more applications to join the committee than ever before. We are grateful to all those who put themselves forward. Our new committee members come from diverse backgrounds and bring a wealth of experience in the data field. They were voted on by you or co-opted to join. Their enthusiasm and passion for data management is truly inspiring, and I have no doubt we will make significant contributions to DAMA UK's mission.

    Looking ahead, the next two years are shaping up to be incredibly important for DAMA UK. Our organisation has grown significantly in recent years and across a diverse range of sectors, both public and private. This reflects the increasing recognition of the importance of the data profession. In an era where data drives decisions across industries, our role as data management professionals has never been more critical. We must continue to evolve and adapt to meet the changing needs of our members and the data community as a whole.

    During the next two years, we will focus on several key priorities:

                1. Membership Growth: We aim to expand our vibrant community, welcoming more professionals into our midst. We’d like to have 3000 members by Dec 2025 to show the strength there is in our profession and continue to learn from each other.

                2. Elevating Industry Standards: Our commitment to raising the bar for our industry includes fostering the growth of certified data professionals and advocating for the recognition of their expertise akin to other professions like accountancy and medicine.

                3. Enhanced Member Support: We recognise the importance of providing unparalleled support to our members, ensuring they have the guidance and resources they need to excel in their careers.

                4. Regular Offerings Review: To stay ahead in the evolving data landscape, we will regularly review and update our offerings to provide the most relevant and up-to-date information, training programs, and resources.

                5. International Collaboration: We value collaboration and knowledge-sharing globally. We seek to foster strong partnerships with other DAMA chapters worldwide to exchange best practices and drive the advancement of data management as a profession.

    I am excited about the journey ahead and the positive changes we can make together.

    In closing, I want to thank each and every one of you, our members, for your continued support. Your passion for data management fuels our collective success, and I am confident that with your dedication, the data community in the UK will continue to grow and to thrive.

    Here's to a promising future for DAMA UK and the data profession at large. Let's embrace the opportunities that lie ahead and work together to make a lasting impact.

    If you’d like to see who is on the committee, you'll find all the great people here.

    Lisa Allen

    Chair, DAMA UK Committee

  • 1 Mar 2023 08:41 | Sue Russell (Administrator)

    Framework for Success: How to transform your organisation for a Data-Driven future - Dave Knifton

    Jenny Andrew, DAMA UK Committee member and head of data at the Chartered Society of Physiotherapy, hosted an entertaining presentation by Dave Knifton, founder of The Data Gym. The webinar was the first in DAMA’s ‘Unleash Your Data’ series with an overarching aim to boost data-led innovation, decision-making, efficiency and agility at organisations.

    We work in distinctly different organisations, but they all have common characteristics. Turning data into a useful resource rather than viewing it as simply a series of problems is a mission that unites all workplaces. But how do you even start to make strides towards a data-driven future that is better for employees and customers alike?

    What ‘unleashing your data’ really means

    During the webinar, Dave explored the foundations of a data-centric strategy that can transform your vision into a deliverable reality. This results in stakeholders having a deep understanding of:

    • the organisation’s data-related functions
    • the relationships between them; and
    • a shared understanding of data’s purpose

    Dave recognises that every organisation is different, and used his presentation to frame these disparities using two types of hypothetical businesses.

    The first is an ‘unenlightened’ organisation. Data activities, processes and professionals are found under one roof. However, these firms don’t embody a complete data strategy. As a consequence they run the risk of simply ‘tinkering’ with data rather than extracting value from it.

    The second type of organisation sits at the other end of the spectrum. It’s a ‘data-centric’ entity that functions differently to its unenlightened counterpart. It does innovative things with its data that move the business along to new areas that might not previously have been considered.

    Despite these obvious differences, Dave argues that common approaches to data management are the only feasible way forward if goals are to be met, and standards and regulations developed and applied across sectors.

    Unleashing the power and value of data creates a wealth of opportunities. However, there is much work to do before organisations reach that point. This begins with determining where your organisation is on its journey to becoming data-centric.

    Viewing business priorities through a data lens

    Data-centric organisations have many key features in common. Largely, these can be divided into three areas, which Dave examined:

    Revenue generation, including data-informed decision-making. This begins with a strong data culture, with employees understanding how to interact with data and the impact it can make. Too many businesses are ‘data inert’ and focus solely on dealing with problematic processes; rather than fixing them but also using data to innovate for growth.

    Operational efficiencies - more agile, cost-effective and sustainable (in both senses of the word). Data literacy is key here, with employees empowered to make evidence-based decisions when using data while having a clear view of intended outcomes. In other words, the whole organisation is built on a foundation of ‘data decision-making by design’. Teams are connected by data; it isn’t hidden in silos; and there’s a line of sight between data input and results, enabling the measurement of positive impact. Automation has a role to play in simplifying and augmenting processes, and thereby achieving operational efficiencies (although the debate about human vs. machine was not a central theme of the presentation).

    Reduced risk of data mismanagement e.g. GDPR non-compliance, and not using data for its intended purpose. Everyone should care that the data being used has the correct characteristics, as Dave calls them, to produce a well-oiled machine of data-controlled outcomes. It also means having a streamlined data estate. This provides legible, understandable datasets that are fit for purpose. Importantly, your firm needs assurance in business-critical matters such as compliance. A data-centric approach will provide this, potentially through a Data Policy Framework.

    A framework for data success

    Achieving success in those three areas described by Dave will afford your organisation ‘sure-footed agility’. But the positive outcomes will only be achieved if organisations realign around data.

    To do that, they must design and implement a Data Assurance Framework that articulates solutions to aspects of data culture, data literacy, management & governance, skills, processes and outcomes.

    This means implementing accessible templates that can be replicated and used across the business. This is true even for teams and individuals who don’t really care about data or how it is employed. The templates should be housed in a toolkit or ‘patterns store’ that all employees can access:

    • Self-serve
    • Model-based
    • Process-driven
    • Discoverable
    • Optimised via feedback

    The benefits of unleashing your data

    Ultimately, Dave believes the framework for a data-centric strategy can relieve the tension that lies between often-misplaced ‘delivery at pace’ that is intended to satisfy stakeholder demands, and a new, considered mindset of ‘delivery principles’ that breed lasting success.

    These principles must be focused on sustainable, sensible outcomes that are driven by improved data management and performance. In this way, data governance, management and business architecture inter-relate harmoniously to enable better processes.

    We’re keen to hear about your own experiences of building a data-centric organisation. Does it align with Dave’s Data Assurance Framework? Or did you take a different approach? Contact DAMA with your thoughts.


  • 1 Nov 2022 13:05 | Sue Russell (Administrator)

    HOW TO TALK TO BUSINESS LEADERS ABOUT DATA MANAGEMENT

    Mark Humphries, DAMA UK Chairman, introduced a lively presentation from Scott Taylor - AKA The Data Whisperer. Scott’s entertaining, inspiring and informative talk tackled head-on the common problems data professionals face when making the business case for data management to leaders at their organisation. How can we frame the tangible benefits while also linking them directly to business goals?

    Every enterprise is focused on transformation to optimise the customer experience (CX). You’ve probably noticed, however, that senior leaders don’t always recognise such a change in strategy must always be underpinned by valuable, structured data. Without it there will be no transformation.

    Scott’s presentation was a call-to-arms for data management professionals, summarising his personal mission to help our industry make the case for ‘better data’. It’s something you’ve probably sought advice about. You may even be building a business case for data management right now.

    Scott recognises it’s rarely as simple as putting together a plan, then presenting it to the senior leadership stakeholders. This can be due to a breakdown in communication (ours is a complex field!) or perhaps a lack of understanding that data is vital for a shift to tech-driven operations. As Scott states: “Digital needs data, and data needs data management.”

    Making your case with The 4Cs

    Transformational CX has many components. Your organisation could be seeking to shift its model to put the customer at the heart of everything it does. Alternatively, the goal might be a switch to direct-to-consumer commerce; a greater focus on self-service; or the need to reassure the target market about aspects of data privacy and consent.

    Of course, data underpins all of these strategic imperatives. Yet it isn’t always available to the required standard, or in a structure that is fit for purpose. Scott says the first step to better CX is defining master data - “All of the data in charge of your business” - in a way that’s easy for busy or baffled boardroom folk to understand. He reveals a good way to encapsulate this is “The 4 Cs” of master data:

    • Code - one for every entity, e.g. product, in your database so it can be easily identified
    • Company - a hierarchy that structures data in use across your organisation e.g. describing a specific client account
    • Category - what kind of data is this - e.g. in which sector or channel - and how can it drive segmentation and analysis?
    • Country - an identifier that directly relates to the geographies where your firm operates, from region to postcode

    In the simplest terms (for non-data experts), data organised around The 4Cs will tell your business who, what and where it relates to, and why it’s unique.

    Let CX trends drive your data management case

    Taking a focus on current macro-level CX trends is key to show no modern business process will work properly without data management. These are the commercial trends the CMO, CTO and CEO and others will likely be aware of already. If you can frame the need for quality, structured master data in these terms it will make your case stronger.

    Here are five trends DAMA has identified as being in play at many organisations:

    • Data mesh - a new approach to data based on distributed architecture for data management
    • Modern data stack - the quest to identify all of the tools needed for a modern approach to data management centred around data science and the data lake
    • Data fabric - a set of data services that is decoupled from any physical platforms to provide seamless access
    • Data ops - a way for data teams to collaborate to improve CX and, ultimately, results
    • ‘Green’ data - the development of data products to help organisations assess and monitor carbon impact

    As Scott says: “Transformation around these and other trends won’t work if you don’t have the data to back it up. That means having a 360-degree view, tying together disparate data sources across platforms. Making it available to the entire enterprise takes a lot of data structuring behind the scenes.”

    Place your data on a three-stage roadmap

    A final piece of the jigsaw when setting out your case is to describe where your organisation is on its transformation journey. Scott identified three stages:

    • Legacy - when data is not in a great state; multiple departments and systems operating in silo. “There isn’t a vertical that isn’t struggling with this,” Scott believes.
    • Integrated - the phase when you’ve begun to take the most important aspects of your operations and placed them at the centre of your business strategy (whether it’s product- or customer-centric).
    • Connected - structured data includes that which links to your partners, described by Scott as “an external network of trusted ecosystems”. This could be any type of intermediary involved in your CX e.g. getting your physical product to market through a distributor; or a digital product or service sent directly to a customer device, perhaps relying on cloud hosting as an intermediary.

    The consistent requirement to move from “silo to centric to network”, as Scott puts it, are authentic identity - trusted data - and a common data structure.

    Putting data to work across your organisation

    If you’re still unsure how to make the case for data management, Scott advocates a focus on the business benefits.

    He uses a framework that he labels “the eight ‘-ates’” to lay out a transformational roadmap, complete with tangible outcomes to make a play for investment in better data: relate (connect with stakeholders); validate (present evidence); integrate (pull sources together); communicate (in clear terms); aggregate (analyse and report); interoperate (connect with third-parties); evaluate (what you need/want); and circulate (data flow drives value).

    We’re keen to hear your own tales of presenting a data management strategy to senior leaders. What were the barriers, and what helped you to succeed?

<< First  < Prev   1   2   3   4   5   ...   Next >  Last >> 

Our Partners and Sponsors







Powered by Wild Apricot Membership Software