How data management is changing

As data and workloads transition to the cloud, data management best practices become increasingly important in guiding how businesses manage the data and use it across the entire organisation.

That’s the view of Sherrine Green-Thompson, SmarterData area lead at open source enterprise solutions company, Obsidian Systems, who says this is especially important in the hybrid cloud environment, and requires a clear understanding not only of how the data is structured, but also of the use cases for data management across the business.

She points out that technology such as DataOps that enables organisations to better analyse their data is becoming more powerful and innovative, placing greater demands on data management plans.

Green-Thompson defines DataOps as “an automated methodology to improve the quality and reduce the cycle time of data analytics”.

“It seeks to make the transition from idea to analytic visualisation or presentation faster and provide decision-makers with more lean data development process features,” she says.

“But despite the automation it provides, DataOps still requires a combination of people, processes, and technology to enable data management solutions to become more agile. It is reliant on a collaborative approach between teams to define the data management strategy and ensure it is integrated across the business. Replacing the silo approach of old with a culture that encourages communication across these teams is vital.”

And what of the future?

Green-Thompson predicts the emergence – or growing prominence – of more advanced data management technologies that are likely to result in changing roles and jobs inside organisations as data becomes increasingly democratised.

“Things such as DaaS (data as a service) will deliver an improved and prioritised focus on privacy, security, and data governance. It will enable the data to be moved easily from one platform to another while reducing redundancy, and providing for easier administration. It will then become a case of users simply subscribing to or pulling data streams as and when the need arises,” she explains.

Another technology she believes will come to the fore is augmented data management which converts metadata from being used for audit, lineage, and reporting only, to powering dynamic systems.

“Augmented data management will see metadata changing from being passive to becoming more active and playing a key role in driving artificial intelligence and machine learning adoption in the business,” she says.

In addition, Green-Thompson believes that increased collaboration throughout the organisation, using DataOps and leveraging multiple technologies such as augmented analytics, event stream processing, optimisation, business rule management, and machine-learning, will result in continuous intelligence for the business.

“Real-time analytics will be integrated within a business operation, processing current and historical data to prescribe actions in response to events. It will provide decision automation or decision support.

“Despite all these changes, data management plans are not going to go away. However, the nature of how the organisation is developing and approaches them is changing, resulting in a more dynamic and agile environment,” she concludes.