With Big Data Comes Big Responsibility - Why Data Management Matters
How many of us have moved into a new house? Have you experienced the thrill of taking ownership and had fun moving furniture around, getting it just as you want it? After a while, you start putting old and disused furniture into the garage, should you be lucky enough to have one, or maybe into a garden shed.
Over time, you add more and more stuff and, one day, you find the garage or shed full. You have no idea where anything is and why things were put in there. Garage owners across the UK would identify with this scenario – being one myself, I completely understand!
The garage analogy
What has this to do with Data Management, you ask? Well, data repositories can get just as cluttered as garages/sheds and this is where the value of Data Management can be truly appreciated. You need to understand what each item is, where it has come from and how valuable it is to you. If more people adopted this practice, how much time and effort could be saved (along with lowering stress levels)?
I wish this were my own original analogy, but it was explained to me by a wise former colleague a few years ago, to give a practical example of why Data Management is needed.
“ The definition of Data Management is provided by DAMA International, the professional organisation for the data management profession - "the development and execution of architectures, policies, practices and procedures that properly manage the full data lifecycle needs of an enterprise. ”
Let's unpick this definition. Every organisation holds data (facts about its structure, membership, finances etc). Any organisation can choose just to hold the data on paper, depending upon its size and complexity. However, once the data becomes significant in size and complexity, that organisation needs to hold it electronically. Usually, this will be in structured databases or (at the very least) on spreadsheets.
Now we have the concept of Big Data to wrestle with as well. This is all about data that can be structured, or unstructured. Unstructured is your emails, photographs, geographic locations and social media interactions. This is usually significantly bigger than structured data, due to the sheer volume and array of formats – this is Big Data. This usually gets "dumped" in what would be called a "Data Lake".
Why do organisations need such data? Well, this gives a rounded, holistic view of the customer and stakeholders and what they are thinking and doing that can be used for commercial gain. Organisations can use this data to make informed strategic decisions and predict the behaviour of customers.
Sorting the wheat from the chaff
OK, so far, so good. Now the fun begins! Big Data is a great concept, but it needs to be managed. Organisations need to know whether the data is of a quality that can be trusted. They need to know if the data is taken from reputable sources. How timely is the data? Are organisations drawing the correct conclusions or is the data just rubbish, clogging up valuable space on servers.This is where Data Management is key.
Organisations need to curate their data and ensure that it is fit for purpose. Therefore, they should employ methods of managing Data Quality (by monitoring and testing the data) and they should ensure that the data has ownership assigned to it. They should be able to clearly work out the journey of that data and map it back to trusted sources. A Data Strategy will pull all of these controls into place and give a roadmap for future developments.
But what is the benefit in monetary value, you may ask? This will vary by organisation and what they hope to achieve (linked to their strategic goals and objectives). However, studies have shown the true value of Big Data.
Data Management used with Big Data = quality outcomes.
By Adam Cox, Senior Data Consultant, Nationwide Data & Analytics - LinkedIn profile