Big data in the water and wastewater industry

Mon 17 Jan 2022

The way big data technologies are used in the water and wastewater industry has changed over the past few years. They are no longer looked at primarily as a means of simply gathering data, but as a management tool for improving operations, performance, and efficiency, according to Mazars’ 2021 US Water Industry Outlook.

Based on a survey that was conducted in 2020/2021, the outlook captures the perspectives of different stakeholders, including water and wastewater systems operators, procurement and other support for water and wastewater companies, government regulators and the investment community.

In line with our findings from earlier surveys, respondents expect new data technologies to most benefit asset management and operational efficiency. Patrick Zerbib, Partner, Data Advisory Services, discusses the benefits of big data for water operations and what organisations need to consider when starting the implementation.

The following is an extract from the 2021 US Water Industry Outlook which can be downloaded here.

What is the overall use of big data across industries, not just water, over the past 2 years? Is the perceived benefit still strong, is data science in high demand, and are companies really using data, or is it just a buzz word?

Over the past few years, digital transformation has spurred an explosion in the volume of data generated across all sectors. The world of Big Data seems to be getting bigger mostly from the rise of mobile devices, applications, wearables, and the Internet of Things (IoT). By 2025, it is estimated that the global datasphere will grow to 175 zettabytes (i.e., 10^21 bytes), with 2 exabytes (10^18 bytes) generated each day. Unstructured and semi-structured data now make up an estimated 80% of that collected by enterprises. Big Data creates significant challenges: information is often siloed, hard to access, and may be unavailable due to multiple legacy systems; data quality, accuracy and integrity may be subpar; and data privacy is a major concern. Higher costs are also associated with larger data mart builds and/or maintenance.

To address these challenges, several infrastructure solutions have emerged:

• Growth of cloud storage: 82% of companies have a mix of legacy and modern cloud technologies, highlighting the rapid transition to the cloud year over year. By 2023, most enterprise companies will have their data stored in a public cloud data warehouse, like Snowflake, Amazon Redshift, and Google Big Query.
• Traditional data warehouses are evolving and being replaced with modern data like solutions. These new architectures are now generally accepted as the target in all industries and are broadly used to enable analytics applications.
• At the same time, many entities are moving away from a monolithic data lake to distributed data mesh/fabric to help overcome scaling issues.

Figure 1 – Which statements regarding big data management applies to your organisation or the water or the industry?

What key areas of water operations can benefit from data management and analysis practices? Can you provide an example?

Faced with aging infrastructure and limited capital to improve it, water operations increasingly require improved efficiency and better cost management, leading to a need for ‘smarter” infrastructure – i.e., one that embeds real- time analytics in the operations.

Potential applications include:

• Real-time water quality monitoring and dashboarding, including turbidity.
• Cost optimisation: optimise plant processes and allocation of resources in the field.
• Asset failure management: preventive maintenance through machine learning modeling.

Figure 2 – Please indicate which two challenges are most likely to benefit from big data analytics.

• Asset management (network) 34%
• Operational efficiency 29%
• Anomaly detection and prevention (end customer) 16%
• Water quality management 11%
• Demand forecasting 10%

The Outlook this year has a major focus on people and training: what is the model for big data, is it more of a people matter or infrastructure? Should big data be accessible to everyone or should it be a tool for a few in the organisation?

Amid a growing need to embed data and analytics in enterprise culture, the journey to becoming a true “data-driven organisation” is challenging. Innovation only happens where people, technology, and process changes meet. If the objective is to build a self-serve infrastructure, water utilities will need to have access to relevant data engineering and science capabilities.

Water professionals often lack the expertise in data engineering and science required to develop relevant machine learning-derived models. Nevertheless, access to technology will need to be supplemented with certain components:

• People: data owners, stewards, producers, and consumers must be designated.
• Technology: master data and metadata management, data quality standards and management, data lifecycle and controls must be put in place.
• Processes: KPIs, data policies/standards/guidelines, and data decision processes must be defined.

Do you believe big data and data analytics can play a role in improving your company’s, or the water and wastewater industry’s operations, performance, and efficiency?

Should organisations that are not yet on this big data journey be concerned about the scope of the undertaking/ investment and how should they get started?

The first step is to conduct a short diagnostic capturing the current baseline. This includes:

• Understanding current data capabilities.
• Understanding the challenges faced by internal users and business partners.
• Understanding organisational requirements for data, process, and technology.

To contain risk and reduce the cost of getting started, best practice is to select one (or two) use case(s) that would help solve a specific challenge identified in the baselining exercise, while providing short to medium term measurable benefits. A good example would be to build a dashboard that provides a new way to monitor a data-rich process and generates alerts that will ultimately enhance decision-making.

The Mazars US Water Industry Outlook is based on a 26-question survey that was conducted in 2020/2021. You can find all the survey results and read the full report here.