ISO/TR 24514:2018
(Main)Activities relating to drinking water and wastewater services — Examples of the use of performance indicators using ISO 24510, ISO 24511 and ISO 24512 and related methodologies
Activities relating to drinking water and wastewater services — Examples of the use of performance indicators using ISO 24510, ISO 24511 and ISO 24512 and related methodologies
This document provides several examples of varying complexity which illustrate the use and intent of the performance assessment methodology set out in ISO 24510, ISO 24511 and ISO 24512. The document also provides examples of the parallel and similar practices for measuring performance or establishing benchmarks as found in various institutional circumstances. These examples represent practices in a range of utilities (e.g. small, medium and large water utilities; water utilities from developed and developing countries; water utilities in both the public and private sectors; government and non-governmental agencies; and utilities with and without certified management systems). They are drawn from many geographical locations.
Activités relatives aux services de l'eau potable et de l'assainissement — Exemples d'utilisation d'indicateurs de performance à l'aide l'ISO 24510, l'ISO 24511 et l'ISO 24512 et des méthodologies associées
General Information
Standards Content (Sample)
TECHNICAL ISO/TR
REPORT 24514
First edition
2018-05
Activities relating to drinking water
and wastewater services — Examples
of the use of performance indicators
using ISO 24510, ISO 24511 and ISO
24512 and related methodologies
Activités relatives aux services de l'eau potable et de
l'assainissement — Exemples d'utilisation d'indicateurs de
performance à l'aide l'ISO 24510, l'ISO 24511 et l'ISO 24512 et des
méthodologies associées
Reference number
ISO/TR 24514:2018(E)
©
ISO 2018
---------------------- Page: 1 ----------------------
ISO/TR 24514:2018(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2018
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2018 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TR 24514:2018(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Format for the examples provided in this document . 3
4.1 General . 3
4.2 Name of the organization . 3
4.3 Background of the organization . 3
4.4 Objective for making use of performance indicators . 3
4.5 Performance indicator methodology . 3
4.6 Summary of the experience . 3
Annex A (informative) International organizations . 4
Annex B (informative) National regulatory organizations . 8
Annex C (informative) Associations or groups of organizations .23
Annex D (informative) Individual utilities .41
Bibliography .65
© ISO 2018 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TR 24514:2018(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www .iso .org/iso/foreword .html.
This document was prepared by Technical Committee ISO/TC 224, Service activities relating to drinking
water supply, wastewater and stormwater systems.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO 2018 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TR 24514:2018(E)
Introduction
0.1 General
This document is a companion document to ISO 24510, ISO 24511 and ISO 24512 and was originally to
be developed with the thought that many organizations (water utilities, regulators, users’ associations)
would benefit from examples illustrating how the performance indicator methodology and the
performance indicators, such as those described and illustrated in the series of standards on the topic of
activities relating to drinking water and wastewater services produced by ISO/TC 224, have been applied
by a variety of organizations. Please refer to ISO 24510, ISO 24511 and ISO 24512 for guidance on the
[1],[2],[3]
process and concepts . However, there are many similar and parallel methodologies measuring
the performance of organizations, not least of which are several “benchmarking” methodologies, some
of which are described in this document. Therefore, this document provides examples of quantification
practices and processes for measuring organizational performance.
Performance indicators as developed by ISO TC 224 are intended to be used primarily within water
utilities, often over time, to demonstrate progress towards achieving high-level corporate objectives. In
practice, these indicators are also being used to indicate projected benefits that could be achieved with
changes in investment strategies or operations. Benchmarking, on the other hand, is used primarily
to demonstrate the efficiency of operations, particularly by sharing information between comparable
organizations often with the identification of best practices related to the particular operation being
benchmarked. Used in a time series, all can also be applied to demonstrating progress towards meeting
objectives and demonstrating continuous improvement. What is confusing is that both can use exactly
the same metric, i.e. a numerator using one data set, and a denominator using another data set. For
example, energy used/megalitre of water produced.
Figure 1 illustrates the basic similarities in the use of a typical metric for the internal uses linked to
achieving corporate objectives (performance indicators) and how benchmarking (as described in
[11]
ISO 24523 ) enables information sharing on metrics and ultimately sharing of best practices. Both
methodologies are intended to assist in continuous improvement and to measure achievement of
objectives.
Figure 1 — Illustration of uses of performance metrics.
Within the field of organizational assessment or measurement, there are several terms that are regarded
at least informally as being interchangeable. These include “performance indicators”, “performance
measures” and “benchmarks”. The first is the preferred terminology of the series of standards on the
topic of activities relating to drinking water and wastewater services produced by ISO/TC 224. The
terminology describing the process for utilizing “performance indicators” and “performance measures”
© ISO 2018 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/TR 24514:2018(E)
tends to be “assessment”. The terminology describing the development and utilization of “benchmarks”
is “benchmarking”. The end result can be the same: both result in the calculation of a metric that
indicates the status achieved in respect of performance of a particular function or activity. However,
these terms are used to achieve subtly different purposes. Performance indicators are normally used
to measure activities within a single organization where the basis of calculation does not change
from year to year and, at least in the ISO standards referenced, are directly related to organizational
objectives at the corporate level.
Benchmarking is intended to encourage and allow comparison between organizations providing similar
services in a defined context. Such methodologies often serve to measure achievement of operational
objectives (which are necessarily a subset of corporate objectives and which support the achievement of
the corporate objectives). That is, the elements comprising the numerator and the elements comprising
the denominator should be identical between the organizations being compared. The benchmarks will
enable comparison of organizations in common and defined fields, but are not necessarily directly
or closely linked to the high-level corporate objectives. ISO 24523 provides information on the
[11]
benchmarking process .
A distinction between the performance indicator methodology and the benchmarking methodology is
that while performance indicator comparison can be an essential part of benchmarking, performance
assessment is a crucial part of benchmarking. Therefore, benchmarking is a way to apply the ISO 2451X
standards. However, “benchmarking” differentiates from pure performance indicator comparisons
through additional and continuing work steps, in particular “analysis” and “implementation” (see
[12]
ISO 24523, Table 1 ), leading to performance improvement.
The purpose of this document is to provide practical examples based on real life applications for the
consideration of utilities using the guidelines in ISO 24510, ISO 24511 and ISO 24512, and also to
indicate examples of where other metrics are employed for regulatory and other purposes.
This document should encourage and assist utilities, particularly small and medium-sized organizations,
when using ISO 24510, ISO 24511 and ISO 24512, to think and communicate clearly about the meaning
and use of performance indicators.
0.2 Summary of the methodology of the series of standards on the topic of activities relating
to drinking water and wastewater services produced by ISO/TC 224
The series of standards on the topic of activities relating to drinking water and wastewater services
produced by ISO/TC 224 provide guidance for water utilities that wish to demonstrate that they are
meeting their broad social and other objectives as established by top management. These often reflect
objectives established implicitly or explicitly in legislation which may govern the delivery of water
services. The series of standards on the topic of activities relating to drinking water and wastewater
services produced by ISO/TC 224 contemplate a three-step process. Step 1 is to determine water
utilities’ strategic objectives. Such objectives in the case of water and sanitation services explicitly
include promoting public health, protecting the environment and providing for a sustainable service.
Step 2 asks what service criteria are to be used to determine if the objective is being met. The final step
asks what metrics should be used to demonstrate that the criteria are being achieved.
For example, Table 1 may represent these three stages in respect to the objective of promoting
public health.
Table 1 — Example of the performance indicator steps employed within the series of standards on
the topic of activities relating to drinking water and wastewater services produced by ISO/TC 224
Objective Service criteria Indicator
(Step 1 – define) (Step 2 – how to measure) (Step 3 – establish a metric)
Promoting public health Delivering safe drinking water Percentage of delivered water quality
tests that meet regulated requirements
vi © ISO 2018 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/TR 24514:2018(E)
0.3 Purpose of the applications
In many countries, governments require that utilities (both public and private) report on their
performance, in a unified and consistent manner. For example, all countries within the OECD are
committed to such a policy and have established requirements for public sector utilities to publish
annual reports indicating measures of their performance. The purpose of this policy and program is
to provide assurances to the population that these utilities are effective in their activities (typically
showing the economic efficiency of the activity in terms of cost/unit of output. For associations
representing member utilities (sometimes in the public sector but often in the private sector) the
association establishes methodologies for reporting on a uniform basis the performances achieved by
the members. The purpose of this may often be to demonstrate good corporate citizenship and may
include indicators of environmental protection, or consumer relations. For individual utilities (again for
both public and private utilities), the purpose of calculating performance measures may be principally
for internal purposes (e.g. reporting to management, demonstrating continuous improvement, or
demonstrating the need for investment in new technology or for repair of infrastructure).
Regardless of the purpose for which the practice of measuring and reporting performance was
established, benefits are obtained and shared with all stakeholders. In addition, all the metrics
developed can be associated with organizational objectives and purposes.
© ISO 2018 – All rights reserved vii
---------------------- Page: 7 ----------------------
TECHNICAL REPORT ISO/TR 24514:2018(E)
Activities relating to drinking water and wastewater
services — Examples of the use of performance indicators
using ISO 24510, ISO 24511 and ISO 24512 and related
methodologies
IMPORTANT — The examples in this report are included only because they illustrate the use
of the series of standards on the topic of activities relating to drinking water and wastewater
services produced by ISO/TC 224 or similar requirements or practices of assessment or
benchmarking.
Value judgements expressed in these examples relate to the relative benefits for each example and
reflect decisions made specifically by the management of the utilities in the examples. No endorsement
is given by ISO/TC 224 to
a) those decisions,
b) the data used, or
c) their conformity with other ISO standards or non-ISO practices or requirements.
No endorsement is given by ISO/TC 224 to any organization or their practices.
No utility’s particular application of the series of standards on the topic of activities relating to drinking
water and wastewater services produced by ISO/TC 224 or the parallel measurement requirements or
practices is recommended because the management of each utility has selected the application most
suited to its needs. No endorsement is given by ISO/TC 224 of the choices made by individual utilities
or the relative merits of these different applications of the series of standards on the topic of activities
relating to drinking water and wastewater services produced by ISO/TC 224.
1 Scope
This document provides several examples of varying complexity which illustrate the use and intent of
the performance assessment methodology set out in ISO 24510, ISO 24511 and ISO 24512. The document
also provides examples of the parallel and similar practices for measuring performance or establishing
benchmarks as found in various institutional circumstances. These examples represent practices in
a range of utilities (e.g. small, medium and large water utilities; water utilities from developed and
developing countries; water utilities in both the public and private sectors; government and non-
governmental agencies; and utilities with and without certified management systems). They are drawn
from many geographical locations.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO 2018 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/TR 24514:2018(E)
3.1
benchmark
single value, representing an accepted reference value derived either from comparisons among
participants or from literature, used for orientation
Note 1 to entry: The benchmark may be determined collaboratively or individually.
Note 2 to entry: By clustering, different benchmarks can occur for different peer groups.
[SOURCE: ISO 24523, 3.2]
3.2
indicator
parameter, or a value derived from parameters, which provides information about a subject matter
with a significance extending beyond that directly associated with a parameter value
Note 1 to entry: Adapted from OECD works on “Core sets of indicators for environmental performance
[11]
reviews” .
Note 2 to entry: Indicators can refer to context, conditions, means, activities or performances.
3.3
measure, noun
value resulting from measurement and the process used to obtain that value
[SOURCE: ISO 9241-11:1998, 3.11]
3.4
performance
measurable result
Note 1 to entry: Performance can relate either to quantitative or qualitative findings.
Note 2 to entry: Performance can relate to the management of activities, processes, products, services, systems
or organizations.
[SOURCE: ISO/IEC Directives Part 1 and Consolidated Supplement — 2017 (8th edition), Annex SL,
Appendix 2, 1605 3.13]
3.5
performance measure
means used to assess the system performance, typically by diagnostic or relative performance methods
[SOURCE: ISO TR 19358:2002, 2.6, modified — term made singular.]
3.6
process
set of interrelated or interacting activities that use inputs to deliver an intended result
Note 1 to entry: Whether the "intended result" of a process is called an output, product or service depends on the
context of the reference.
Note 2 to entry: Inputs to a process are generally the outputs of other processes and outputs of a process are
generally the inputs to other processes.
Note 3 to entry: Two or more interrelated and interacting processes in series can also be referred to as a process.
Note 4 to entry: Processes in an organization are generally planned and carried out under controlled conditions
to add value.
Note 5 to entry: A process where the conformity of the resulting output cannot be readily or economically
validated is frequently referred to as a "special process".
2 © ISO 2018 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/TR 24514:2018(E)
Note 6 to entry: In benchmarking, organizational and technical processes and combinations of both of them are
considered. A process within the meaning of benchmarking comprises a combination of one task with one plant/
one object (e.g. operate sewer network, treat wastewater, treat drinking water, provide domestic connection,
further train staff, purchase material).
[SOURCE: ISO 9000: 2015, 3.4.1, modified — Note 6 to entry replaced.]
4 Format for the examples provided in this document
4.1 General
Examples illustrating the application of the series of standards on the topic of activities relating
to drinking water and wastewater services produced by ISO/TC 224 or the parallel alternative
methodologies are provided in Annexes A to D. The contents of these examples are generally organized
under the headings given from 4.2 to 4.5. However, where full information on the practice or requirement
is not available at the time of publication of this document, summary information and a bibliographical
reference have been supplied.
Please note, the Annexes distinguish between methodologies and requirements established by
international organizations (Annex A), national regulatory bodies (Annex B), associations or groups of
utilities (Annex C), or individual utilities (Annex D).
4.2 Name of the organization
This section simply provides the identity of the organization offering its experience.
4.3 Background of the organization
This section briefly describes the organization and its history. The purpose is to provide a context
for understanding how the performance assessment methodologies of the series of standards on the
topic of activities relating to drinking water and wastewater services produced by ISO/TC 224, or
similar requirements or practices of assessment or benchmarking, were applied. It is also intended to
provide suitable information for a reader of this document to relate his or her own organization to the
organization described in the example.
4.4 Objective for making use of performance indicators
This section includes a description of the purpose of the organization when implementing the
requirement to develop and report the use of performance indicators whether conforming to the series
of standards on the topic of activities relating to drinking water and wastewater services produced by
ISO/TC 224 or similar requirements or practices of assessment or benchmarking.
4.5 Performance indicator methodology
This section provides a short description of each step followed in the methodology. Tables and figures
may be attached to illustrate the process.
4.6 Summary of the experience
This section includes a summary of any lessons learned by the organization from the production of
performance indicators whether conforming to the series of standards on the topic of activities
relating to drinking water and wastewater services produced by ISO/TC 224 or other similar practices
of assessment or benchmarking applied, as the case may be. It may also include the organization’s
conclusions on its future expanded application of the practice of assessment or benchmarking.
© ISO 2018 – All rights reserved 3
---------------------- Page: 10 ----------------------
ISO/TR 24514:2018(E)
Annex A
(informative)
International organizations
1)
A.1 European Environment Agency — Purposes of performance indicators
European policies are increasingly focused on preserving the Earth's limited resources in a sustainable
manner, while minimizing impacts on the environment. This is included in the resource efficiency
and green economy agendas. In order to obtain knowledge on the actual pressure on the aquatic
environment from water abstractions and emission of pollutants, and for assessing urban water
management, we need to extend the knowledge base beyond compliance with current legislation. With
the aim of improving the European level knowledge base in urban water management in the resource
efficiency context, the European Environment Agency (EEA) hosted an expert meeting jointly organized
with leading water associations in Europe. This event framed the context and discussed topics related
to the exploitation of data already available with water utility associations and benchmarking networks
beyond what is currently reported via institutional frameworks for implementing legislation.
EU-level assessments of the resource efficiency or environmental performance of water utilities are
not currently as holistic as they could be. There is considerable reporting of environmental data
concerning water already in place, from the local to the EU level. However, these reporting obligations
are primarily concerned with the water quality parameters applicable to drinking water and treated
urban wastewater. The parameters are related to compliance with the EU directives pertaining to the
achievement of drinking water standards, urban wastewater collection and treatment requirements,
and receiving water quality objectives.
[5]
As outlined in the EEA report ‘Towards the efficient use of water resources in Europe’ economic
production cannot be sustained if it requires excessive water use and burdens natural resources. It
is thus essential that water uses and efficiencies are also considered in water management practices,
including: the actual pressures in the aquatic environment from water abstractions, the resulting
emissions of pollutants, and the energy consumption/recovery from managing the urban water cycle.
This report follows on from the discussions in the expert meeting on how the organizations and networks
involved in urban water management can share their knowledge bases to support environmental and
resource efficiency policies, and technical improvements. The availability of this knowledge base could
create a more comprehensive approach to assessing Europe's water resources and threats. It could also
enable a comparison of the environmental performance of different water utilities, monitor progress
over time, and aid the implementation of novel environmental technologies.
In its response to the European Citizen's Initiative “Right2Water”, the European Commission committed
to exploring the idea of benchmarking water quality and will cooperate with existing initiatives to
provide a wider set of benchmarks for water services. This significantly contributes to improving the
transparency and accountability of water service providers by giving citizens access to comparable data
on the key economic, technical and quality performance indicators of water operators. The information
provided in this report, although having a specific focus on environmental performance based on data
from voluntary benchmarking exercises, can be a useful contribution to this debate.
1) The following texts (but not the subheadings) are excerpts from the Executive Summary of the EEA Technical
[4]
Report No. 5/2014 .
4 © ISO 2018 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/TR 24514:2018(E)
A.2 International Benchmarking Network of Water and Wastewater Utilities
(IBNET) of the World Bank Group, Global Water Practice (www .ib -net .org)
A.2.1 Background of the organization
The World Bank is a development institution providing financial resources for development in more
than 150 countries and territories. The World Bank Water Global Practice is in charge of development,
implementation, supervision and assessment of projects related to water. The investment directly into
water and sanitation utilities constitutes roughly 60 % of Water Global Practice operations and is just
above one billion dollars a year for the last 10 years.
Performance and search for good utilities was always a key topic of interest of the World Bank activities
in the water sector. In order to do that, the World Bank established the International Benchmarking
Network of Water and Wastewater Utilities (IBNET) within the Bank’s Water and Sanitation Program.
Client governments develop standard indicators and maintain data to efficiently allocate financial
resources and develop sector improvement programs that address water and wastewater services for
all consumers, including the poor. IBNET provides a fact-based framework to measure the performance
of a service provider and set the standard for water sector performance assessment. IBNET provides
governments, utilities and the public with a clear and objective picture of water services and utility
performance.
A.2.2 Objective for makin
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.