Who should measure the Sustainable Development Goals?

Amid conversations about indicators for measuring progress on the post-2015 Sustainable Development Goals, Saferworld’s Thomas Wheeler explores the question of who should be producing the data that these global indicators will rely on. He explores the benefits of using national statistical systems to measure progress, but also makes the case for third parties playing a significant role.

Saferworld has been working with UNDP and others to bring together experts to help develop global indicators to monitor progress on peace, justice and governance issues under Goal 16 of the proposed Sustainable Development Goals (SDGs).

With a final report forthcoming, this expert group recently met in Vienna, with an impressive spread of participants from National Statistics Offices (NSOs), violence observatories, polling companies and research organisations – all of whom produce data which is highly relevant to the indicators being developed.

There were plenty of issues to discuss: given pressures to cap the number of global indicators to no more than one per target (or less!), which indicators to prioritise, and how? How best to balance between different types of indicators? Should the group recommend different types of data disaggregation for different targets – or just push for maximum disaggregation across all targets?

One issue stood out: the role of those outside official national statistical systems in producing the data that global indicators will rely on. These 'third parties' include multilateral agencies, NGOs, polling companies, research organisations and even citizens themselves. Take, for example, an indicator on the percentage of people who have paid bribes to officials: must we rely on NSOs to produce this number, or can we draw on data already produced by Transparency International, which in 2013 produced data on this for 107 countries? What about the percentage of people who feel safe walking home at night? Must we wait for official sources to start producing this data more widely, or can we draw on resources such as the Gallup World Poll, which asks this question in 140 countries every year?

Listening to the current discussions on indicators, the focus seems to be overwhelmingly on NSOs. For example, a recent report from the Sustainable Development Solutions Network on estimated costs for producing SDG data restricts its focus to national statistical systems. The members of the UN's Inter-Agency Expert Working Group (IAEG) – formed to agree on the final set of global indicators – will be drawn exclusively from NSOs. So a process has gathered steam in which states are set to agree with each other that they should each be monitored by themselves.  

Several assumptions underpin the belief that NSOs should play a significant role in producing SDG data over the next 15 years:

1. NSOs have more experience than others in gathering data that is relevant for the SDGs and Goal 16, including for example on administrative issues.

2. Data on the development of our societies is a public good: it makes sense that public bodies produce it and that we build their capacity to do so.

3. NSOs follow the highest methodological standards. They're staffed by professional statisticians.

4. Data about a country should be produced and owned by that country. This ownership increases the legitimacy of the data – and the likelihood of its use by policymakers.

5. The state will play a central role in driving national-level progress towards meeting the SDGs. Officials require data to guide their policy making, official bodies should be responsible for gathering it.

For the most part, the reasoning behind most of these arguments is sound. However, there are drawbacks to a model that overly relies on NSOs, and a number of reasons why their efforts to measure the SDGs should be complemented by third parties:

• Filling data gaps: Many NSOs don't have enough capacity to monitor the 169 proposed SDG targets – far from it in many parts of the world. This is why the number of global indicators is being restricted, which risks undermining the integrity of the whole framework if only some of the global targets are globally monitored. While one solution might be to consolidate the number of targets, by using data from third parties we can take the burden off NSOs. We can thus ensure that we’re measuring all the issues member states have agreed should be included.

• Applying multi-stakeholder principles consistently: The private sector is being called upon to help finance the SDGs. UN agencies and civil society groups are being called up for help to implement them. Why should this multi-stakeholder approach not apply to monitoring?

• Driving innovation: We desperately need innovation to deepen, widen and cheapen data availability. Innovation will be richest when it involves a diverse range of actors working together as part of an open and dynamic ecosystem of data production. NSOs cannot deliver such a data revolution on their own.

• Methodological rigour: If third parties follow exactly the same methodological standards as NSOs – and are open to similar levels of scrutiny – then there is every reason to view their data as equally valid.

• Broad ownership of data: Country ownership is about much more than state ownership. Pluralistic data production means data ownership across society. Furthermore, in some countries NSOs are not as impartial as international principles demand that they should be; they're often influenced by political pressure. Concentrating control over the production of data in the hands of state bodies is a particular risk in a world in which we know that some people are persecuted for who they are – especially when this data is disaggregated by social identity.

• Accountability: Policymakers not only need data to make decisions, but civil society, opposition politicians, activists and the media need it to hold them to account. Third party data offers a crucial check and balance that can help ensure that official data portrays the genuine reality within society. More broadly, especially when it comes to issues like justice, the rule of law or human rights, should official bodies be given the sole responsibility for monitoring state performance? Use of a balanced range of sources could be important to build public trust and credibility in the SDGs and how they are being monitored.

The group that met in Vienna explored the role of third parties with an open mind and touched on many of these issues. Very encouragingly, it was the NSOs at our meeting who, drawing on their own experiences, made some of the most striking arguments in favour of third parties playing an active role. Let’s hope that these views are now duly raised in the upcoming IAEG group on indicators.

Thomas Wheeler is a Conflict and Security Adviser at Saferworld.

Read more about our work on the post-2015 development agenda.

“Third party data offers a crucial check and balance that can help ensure that official data portrays the genuine reality within society.”

Thomas Wheeler