Introduction

There’s no question that data is becoming an important topic in almost every organisation. For some organisations data will be seen as a challenge – as security breaches, privacy violations and data misuse appear on the news almost every day; for other organisations, data is an opportunity as new startups look to use data in new ways and established organisations look to monetise their current data. What is common across all organisations is the recognition that data cannot be ignored – data doesn’t look after itself and the value inherent in data cannot be realised without both up-front investment and ongoing management.

If organisations are to deal with the challenges and realise the strategic value of data then they will need to look at how data is managed in a new light.

A term in common use is that data must be seen as an “asset”; but what does this mean, and how do we look after a data “asset”? Let’s look these questions in a little more detail.

Data as an Asset

To help us think about data as an asset, let’s use an analogy. One of the oldest types of asset that humans have created are buildings, and we have a mature and pretty complete idea on how buildings are managed as an asset, so we can examine those best practices and draw some lessons to be applied to data.

Let’s think about a large company that has grown quickly over time, both organically and by acquisition. It has acquired a wide variety of buildings – without an overseeing strategy – and all their buildings are different. They all look different; they have different, bespoke maintenance schedules; their entrance security systems are different; they all have different refreshment capabilities, their disabled access is unclear; their leases are all different; the furniture is different with different suppliers and so on …

Any mature organisation would find this unacceptable and it would be prepared to put in significant investment to make sure their buildings were well managed – even if the business case doesn’t show a clear short-term return.

In data terms, this is exactly where most organisations find themselves today. They have hundreds, maybe thousands, of databases with data locked inside. Many organisations don’t even know what data they have and where it is, let alone have a strategy to manage the data. The recent trend towards using a wider variety of database types and suppliers actually makes the situation worse – in a race to make a developer’s life easy, we have made it significantly harder to manage our data. We have allowed the builders to choose their own bespoke and unique materials with little thought about future maintenance of the buildings.

What organisations need is a clear set of policies and strategies that are based not only on the build cost, but on the full lifetime cost, and one that aims at releasing the value in data, rather than it being hidden away.

Data Platform

The most effective way that we have today of managing the data asset is through the concept of Data Platform. At the most fundamental level, this is a means by which the necessary policies and database strategies to drive good data practices can be enacted and enforced. It provides both a framework for managing data and a technical platform for delivering databases themselves.

In our analogy, the Data Platform provides the building standards, architecture and designs, plus the actual standardised building materials themselves. We are at risk of stretching our analogy too far, but one of the clear differences between data systems and buildings is in the initial start-up cost; getting a new building to the point where it is usable will take a very significant investment and many months or years – whereas getting a database system up and running can often be done for negligible cost and in minutes.

As a result, in the world of data, we will need stronger governance and enforcement as we cannot rely on financial and budget controls to govern the creation of new databases. This is a key reason why the data platform must be realised as a carefully managed platform capability, and not just as a set of principles and a strategy.

Data Platform Drivers

Let’s look at the typical drivers and principles of a data platform.

The typical drivers that we see mentioned by large organisations are :

These drivers are very common – in fact, in many organisations, it is the first of these – faster deployment – that tends to dominate driven by infrastructure teams. But with this narrow view, the data platform is no more than an automated way to create databases on-demand via a GUI; there is very little post-deployment management of the database and the platform is used solely for rapid deployment of development and test databases. For example, the open source DBaaS project, Trove, does little more than the initial setup of a database and database backup.

A few enlightened organisations, however, take a much more strategic view of their data and embrace the concept of data as an asset, and so they also acknowledge five additional drivers

• To Improve understanding of the data
• To Improve control of data
• To Improve visibility and accessibility of data
• To reduce the risk of a security breach
• To reduce the risk of data corruption or loss of data
  

The value of a data platform service to an organisation can be increased dramatically if these drivers can be addressed by the service, and this makes the business justification for the adoption of data platform much stronger. The good news is that a well-designed platform will automatically deliver many of these, and where there are gaps, these can be addressed at little extra cost and time if addressed early enough in the design phase. Let’s look at what extra is needed.

Service Design

When we build a data platform, there are five fundamental principles that enable us to deliver against those drivers;

• Simplification
• Service Standardisation
• Optimisation
• Automation
• Security in depth
  

To support the data-as-an-asset approach, we need to complement these with two new principles adopted from best practices in Data Management. It is often the case that “Data Management” and “Data Platform” are seen as entirely different areas of expertise, however, if we build into our platform some of the principles of good Data Management, then the benefits can be very significant. The two most important areas we must include are :

• Data Ownership
• Data Catalogue
  

To deliver Data Ownership we need do not much more than add an extra governance principle – i.e. every dataset brought onto the platform, or every new dataset created must have an owner. For the Data Catalogue, it is not much more complex we simply must ensure that all the datasets are catalogued, and that catalogue must be accessible. There are many tools in the marketplace that can do this today, and by applying to every dataset from the outset, then we can gradually build up a complete view over the entire data landscape.

It’s worth noting that the Data Catalogue capability need not be limited to the datasets in the platform, it can catalogue data anywhere, but if delivered as part of the platform program, and its use mandated from the start, then it will naturally grow to become an authoritative data catalogue.

Conclusion

Database-as-a-service is recognised as an effective means to manage database at lower cost and with more agility. By extending the concept of a data platform to embrace Data Management principles, we can turn a technical capability into a strategic Data Asset Platform.