Together let’s build
a sustainable future


Big data management can offer insights of how to reliably provide services and efficiently maintain assets

By: Mohamed Alaaeldin Mohamed, Mechanical Engineer, Chair of QGBC’s Water interest Group

Published on: November 8th, 2017

Every digital process produces data. Every second, millions of new email exchanges, Google searches, photo captures and metre readings proliferate large, complex data sets. The world is racing to uncharted territories as data are exploding exponentially. Naturally, the discipline of Big Data has emerged as an analytical and managerial tool to maximise benefits of the increasingly ubiquitous large data sets.

Before the competence of big data management grew to be the key competitive advantage for many businesses, big data analytics had been siloed in the academic study of Statistical Analysis. Outside of academe, big data had limited applications in areas such as marketing/advertising and the financial market. In the past decade, however, big data management has traversed to a wider range of applications including utilities infrastructure.

Big data analytics advance the utility operation, asset management and development planning

In network operations, demand management, first and foremost, establishes how water resources and supplies are managed. Similar to marrying advertisers’ messages to collected consumer preferences, analysing the data streams of utility customer demographics, costumer relationships, consumption rates and billing records can quantify the water demand. Daily in Qatar, for example, more than 2 million cubic metres of water are produced and distributed to 2.4 million people in 9 municipalities. With various water uses and consumption rates, it is imperative to have a clear demand management plan to address overly consumptive uses and avoid the need to draw on new water resources.
Water leaked and unaccounted for– through theft or metring inaccuracies– in the supply network is a major operations problem. Net water losses can be identified under Non-Revenue Water (NRW): that is the difference between water pumped into the distribution network and the actual amount of water metred by consumers. In addition to wasting a precious finite resource, NRW financially burdens governments and utility providers. NRW is a pressing problem in water infrastructures, especially in developing countries where it can reach as high as 40% of the water supply. Equipping the water utility with big data will enable continuous auditing for sources of NRW which can then be quickly identified and resolved.

Additionally, water utility operations steered by big data can:
  • establish credible consumption baselines by peer-to-peer comparisons;
  • enact targeted consumer conservation programs, incentive programs and rebate programs;
  • provide added-value proactive/pre-emptive services such as billing error identification;
  • maintain standard water quality; and
  • improve the utility’s emergency response and service restoration time for incidents ranging from pipe bursts to floods by coordinating scenario and contingency planning.

Water data analytics can also be a powerful tool for asset management. With the concurrent advancements in the geospatial monitoring technology, data analytics can assist in maintenance, repair and replacement of network components. Water data analytics can also help:
  • understand thoroughly how different types of valves function in various network topologies, i.e. loops, branches and ends;
  • optimise field workload and workforce resourcing;
  • classify sensor and analyser placement techniques; and
  • identify equipment inconsistencies in what could be millions of metres of mains and connections.

In 2015, 55 thousand metres of water connections were maintained and/or replaced in Qatar. 685 thousand metres of new mains, ranging from 0.1 to 2.4 metres in diameter, were also laid in the same year. Qatar adds, on average, 65 thousand metres of new mains yearly to service the sprawling urbanisation.

Prudent planning of infrastructure is a pillar of sustainable urbanisation. Water treatment, supply mains, sewer works and wastewater treatment plants are all major parts of a city’s critical infrastructure which obviously require copious planning. Combining disparate data on operations with data on asset management is key for strategic planning of water and wastewater utilities. Inasmuch as made available by historical records, data can be assimilated to project water demand, asset attrition rate, network re/zoning and land use for either expansion or new development planning.

Advanced Metring Infrastructure is all of the hardware and software measuring and transmitting data on the utility to the service provider

Digitisation of the utility infrastructure is required for data generation by deploying smart metres and constructing what is known as the Advanced Metring Infrastructure (AMI). Smart metres can orderly record and transmit large number of service metrics as well as information on the condition of the network’s physical assets. The currently predominant digitisation technologies, such as Supervisory Control and Data Acquisition (SCADA), collect measurements, mainly voltage and current, approximately every 15 minutes. However, new technologies, such as Phasor Monitoring Units (PMUs), are pushing the needle on sophistication. PMUs can collect measurements up to 30 times per second, which clearly puts the ‘big’ in big data.

Following on from AMI, harvesting useful information from a myriad of data necessitates ‘Big Data Analytics’ capabilities which begin by understanding the nature of available data. Discrete data are characterised by their volume, variety, velocity and veracity. Recognising patterns, correlations, trends and preferences– Data Mining– is how Big Data specialists uncover information from streams of raw data. Incentives and benefits on both sides– utility providers and customers– may then be synchronised by mathematical models, such as those driven from Stochastic Process, Bayesian Inference and Game Theory. The knowledge from big data, about utility is lastly disseminated to field operators, engineers and policy makers.

Volume: amount of raw data; Variety: different types of data; Velocity: speed of data streaming and processing; Veracity: accuracy and soundness of data

Kahramaa has completed a pilot AMI project in 2015 for the electric grid in an effort to roll out AMI technology. The project covered 17,000 residential and commercial customers in 3 different zones in Doha. Momentum to deploy AMI for the water utility in Qatar is yet to build up.

Big Data can also pose some challenges:
  • Especially to a newly adopting industry, trying to recognise which directions of analysis are useful to pursue, and which are not, can be prohibitively overwhelming.
  • On the extreme end, being ‘too data-driven’ – the problem known as the Big Data Dilemma – can overpower technical/engineering intuition or have long-term results overridden by short-term ones. It is important to verify data analysis results and recognise outliers or invalid measurements which in some cases can be propagated by data analytics software.
  • Data privacy and information security in the cyberspace is a concern for both costumers and utility providers. Any cyberspace is as secure as its weakest link. Thus, protection against infringement requires comprehensive strategies that combine definite industry principles, enforced governmental policies, and obviously, up-to-date software tools to safeguard IT networks.

Smart water projects, including upgrades to AMI and big data analytics, are growing in number especially in the US and the UK, yet there is ample room for innovation. Data analytics can be very beneficial for water networks with added complexities, such as district cooling (or heating) networks. In arid climates, district cooling is indispensable in the built environment. Qatar consumes 7 million cubic meters of water yearly to generate 500 thousand tons of refrigeration (1.8 gigawatts), making water management as important as energy management. Even in water supply and wastewater treatment, energy remains the second most major operations cost factor after labour cost. A water-energy informatics platform is essential to provide a high-resolution image of the utility performance.

Getting smart about water conservation is equally incumbent upon utility providers as well as consumers. The tool of big data analytics in the hands of utility providers can supplant precarious personal judgement and human error with data-informed decisions. As Qatar’s utilities infrastructure is still young and growing, now is the time, especially for the water utility, to shift the centre of gravity from the mere physical network to the wealth of data.