CHAPTER VIII

REPORTING AND USING THE SURVEY FINDINGS

This chapter is for programme decision-makers, survey coordinators and techni-

cal resource persons. It tells you how to:

U Present the findings to decision-makers and the general public.

U Produce a preliminary report.
U Produce an executive summary.
U Produce a full report.
U Use your results. (Illustrations are provided.)

REPORTING AT MID-DECADE
The report at mid-decade by the Secretary-General of the United Nations is important because it will
summarise the way the international community has joined hands with governments, and they in turn
with their constituents, to see that the promises made at the World Summit for Children engender
action. The very act of measuring and reviewing progress toward the goals of each National
Programme of Action is as important as achieving the goals themselves.
Each survey, from start to finish, is an opportunity for raising awareness. The measures of
the mid-decade goal indicators should be used to reinforce programmes at all levels, to redirect or
make corrective adjustments to programmes and National Plans of Action where necessary and to
help policy makers take action that will build and sustain progress. That is why it is necessary to use
the survey results to the fullest extent, building on the plans made in the early stages of survey
planning.

PRESENTING FINDINGS TO THE GENERAL PUBLIC

You and your team may be under intense pressure to release results quickly. Government officials,
international organizations, statistical offices and even journalists will be asking, "What are the
numbers?" These numbers, whether they are correct or not, will take on a life of their own once
released, and may cause you a lot of trouble if you have not checked your arithmetic carefully.
During the course of the field work you should have used as many occasions as possible to
report first impressions, to raise questions and to start corrective action at all levels. For example,
you might ask the following questions: "Why isn’t iodized salt found in the households?" "Why is

8.1
8.2 MULTIPLE-INDICATOR SURVEY HANDBOOK

the vitamin A programme not reaching this area?" "Why are so many fewer girls attending school
than reported by the school inspectors?"
Written summaries of overall results require more care. Once you have looked at your data
critically, made the necessary calculations to produce your estimates and, if appropriate, plotted your
results on graphs, you must produce reports that give "the numbers." These may take different forms,
including policy briefs and summary sheets as well as reports.
Aim to get findings out quickly once you and your team have checked the results and tables.
To speed results into action, plan on producing two reports: a preliminary report followed by a full
technical report (which should include an executive summary).
Dialogue between health workers and community leaders can be encouraged by continued
discussion around each report generated by this multiple-indicator survey. Even if subnational
estimates are not reported, the national results can provide a focal point for discussing local or
district-level problems and successes and for prioritizing future activities.
Raising public awareness of special problem areas (either specific indicators or areas at high
risk) can lead to positive responses if the results are presented skillfully. And public understanding
of problems can often be an important leverage tool, especially when programmes are community-
based.

Producing a Preliminary Report

The preliminary report is a brief and early version of the full technical report. It contains:

3 the aims and objectives of the survey

3 major results
3 a discussion and preliminary recommendations
3 the sample and methodology
3 the field work: what questions were asked, how they were asked, and who asked them

Prepare the preliminary report first. Most sections of this brief report can be written as the
survey operation progresses. Once the data are analysed, all that will remain will be to add the
estimates of each indicator. Include only those major findings that are firm.

L It is important to have a section on the methodology even in the preliminary report. This
section should describe the sampling scheme, the instruments, the field process, number of
interviewers, losses and refusals and so forth. Position the methodology section in the report
so that it does not divert the attention of policy makers away from the findings themselves.
REPORTING AND USING THE SURVEY FINDINGS 8.3

Select the main findings and recommendations that you want everyone to remember. Use the
preliminary report to circulate your main findings&some of which may require judgement and
subsequent discussion&while there is still interest in the survey. If you have planned well and your
survey organization runs smoothly, you should be able to prepare the preliminary report within one
month to six weeks following the end of field work.
You may use the publication of the preliminary report as a vehicle to stimulate interest in the
current health situation among government agencies, nongovernmental organizations (NGOs), other
multilateral donors, the press and the public. You can also show where the biggest data gaps existed
before the survey was conducted, and how these results have contributed to updating old
information.
All major participants in the survey who are involved in the planning, implementation or
analysis stages should be invited to participate in a meeting to present and discuss the main findings.
This publicity may result in additional interest in the survey results and lead to additional
analysis of the data. Once the full technical report is complete, the survey data should be made
available to qualified and interested parties for further use. This can contribute to the effective and
widespread use of the survey results, and multiply the benefits of your effort and expenditure.
Other forms of survey reports may also be useful for reaching certain audiences: a short
summary for those who were involved with the survey at some stage, such as interviewers or those
who have given support to the survey; a report for community leaders about services and problems
identified, including the most important conclusions of the survey analysis; and a press release
summarising the most important conclusions of the survey. It is important to ensure that the press
correctly reports the survey findings and does not distort them. Preparing your own press release can
help ensure that results are accurately reported.

The Full Survey Report: Presenting Findings to Decision-Makers at Different Levels

The Executive Summary

You and your national counterparts should agree upon the recommendations and actions to be taken
upon publication of the final survey report. At this stage your findings are firm, and you and your
team have had time to discuss and plan how to use them. This marks the start of advocacy efforts,
programme consultations and strengthening and re-deploying resources.
Your full technical report should begin with an executive summary (see Box 8.1). Keep the
executive summary short; it should not exceed a few pages, because policy makers will not read
more than that.
The summary should include more descriptive and comparative information than technical
information. It should report the highlights of the survey, but you must be sure of your findings since
8.4 MULTIPLE-INDICATOR SURVEY HANDBOOK

policy decisions will be made on the basis of this part of the report.

O EXAMPLE:
Policy makers will want to know how the new estimates compare with past
estimates. Has the country improved? How does it compare with other countries?

Remember that diagrams have more impact than long tables.

O EXAMPLE:

The summary is also a tool for introducing the mid-decade goal indicators to a wide audience
and can be used to emphasise the quality of previous information available about these indicators
to policy makers. In many countries, some indicators have never been measured at the national level
(salt iodization may be one example). Your presentation of results to policy makers should make this
clear. The presentation should help to stimulate debate and lead to programmatic decisions where
results indicate that action needs to be taken.
You may also be able to arrange special meetings on specific topics to which the survey
results call attention.

O EXAMPLE:
Legislation to promote salt iodization has been on the books for many years, and policy
makers thought that public concern about goitre was unwarranted. The survey shows low
levels of iodized salt in households&and that finding offers an opportunity to focus new
attention on iodization needs and remedial action.
REPORTING AND USING THE SURVEY FINDINGS 8.5

Box 8.1
ITEMS TO INCLUDE IN AN EXECUTIVE SUMMARY
R Purpose of the survey
R Summary of progress in achieving goals, according to indicators measured by
the survey
R Definitions of each indicator
R A brief description of the survey steps
R Country trends for goal indicators that have past data for comparison
R The main recommendations

Producing the Technical Report

The full technical report is essential if your survey results are to be viewed from an international
perspective. This report enables all readers to judge the technical aspects of the survey operation, and
to evaluate for themselves the quality of the data it produced and upon which your results are based.
These surveys to monitor indicators of the Mid-Decade Goals must produce results that can
withstand the heat of intense scrutiny and international comparison. For this reason, it is important
to give all interested parties the information necessary to evaluate your results (see Box 8.2). That
is the purpose of a full survey report.
The technical report is also the document governments will subsequently share for
international reporting. Your role as an advisor is important for encouraging governments to share
their results. An emphasis on process rather than on achievements should balance pride in
accomplishment with an interest in turning problems into reasons for creating alliances for action.
These alliances can be forged at many levels, including regional and international ones, when reports
are shared openly. As soon as possible, copies of the report should be transmitted to all interested
parties, including your UNICEF Regional Office and UNICEF Headquarters.
Be sure to include a discussion of the quality of the data the survey produced. Report the
number of households sampled that did not complete the questionnaire, indicating the reasons for
nonresponse. Indicate which items in the questionnaire were particularly difficult for respondents
to answer, indicating those with high proportions of respondents answering "don’t know" or items
with a high proportion of responses coded as "missing." Also, describe your survey sample of
households, women and children according to the sociodemographic characteristics you collected.
For example, describe the per cent urban and rural, the average number of persons per household,
the sex and age distribution of children and any other characteristics of the sample you think are
important. Describe the details of the sample design in an appendix to the main report.
8.6 MULTIPLE-INDICATOR SURVEY HANDBOOK

Box 8.2
THE COMPONENTS OF A FULL SURVEY REPORT

3 Executive summary.
3 Aims and objectives of the survey.
3 Sample design and size, what was used as a sampling frame, and the method used
to choose households in each community.
3 The questionnaire.
3 Description of the field work.
3 A description of when and how the field work was done and a list of the supervisors
and interviewers, including a copy of the questionnaire, along with interviewers’ and
supervisors’ instructions.
3 Data entry procedures:
— Was data cleaning and editing necessary? What was done?
— How much time and how many personnel were needed?
3 Evaluation of data quality, including a description of problems with missing data for
any particular questions and the nonresponse rate.
3 Results and interpretation of your estimates.
— Where possible, compare your results with those obtained from other sources
and with neighboring countries.
3 Conclusions and actions planned, recommendations for future surveys or studies,
acknowledgements for help and support received from individuals and
organizations.

Report the Indicator Estimates

Describe briefly each method of calculation and the data used in the calculations. Report the values
of each indicator. Explain and report the margin of error around your estimates. Caution your readers
that all the estimates obtained fall within these confidence limits.
Use tables, graphs and charts to display your results.

O EXAMPLE:
Include a table giving low-weight-for-age data, as in the following illustrative table:
REPORTING AND USING THE SURVEY FINDINGS 8.7

Illustrative Table 8.1 Per cent of under-fives with low

weight-for-age (and margins of error), urban and rural
areas
Number of
Residence Low weight-for-age (%)* children
Urban 10.7 (9.5, 12.5) 1,273
Rural 15.9 (13.0, 16.7) 1,989
Note: Children are classified as low weight-for-age if their z-score
was two or more standard deviations below the median interna-
tional standard.

O EXAMPLE:
Present a graph showing the changes in the amount of fluids and food that mothers
give to children who have diarrhoea.

Report estimates for subpopulations if your sample size allows these calculations.

O EXAMPLE:
Report estimates for males and females separately, for urban and rural areas sepa-
rately (if you have distinguished rural from urban clusters) and for regions.

Compare your results with any earlier estimates and reports which are available to you. Read
reports from other surveys before trying to interpret your results and point out any caveats about the
comparisons being made.
8.8 MULTIPLE-INDICATOR SURVEY HANDBOOK

O EXAMPLE:
Present a table giving information about diarrhoea prevalence.
Illustrative Table 2. Incidence of diarrhoea by child’s sex
and residence, and per cent of children under five years of
age who had diarrhoea in the two weeks preceding the
survey
Per cent with Number of
Variable diarrhoea children

Sex
Male 27.5% 2,582

Female 28.7% 2,580

Residence
Urban 28.1% 2,535

Rural 28.1% 2,626

Finally, be sure to circulate an early draft of the report to all collaborators and others who
may be able to provide valuable comments before it is finally published.

FROM DATA TO INFORMATION

Using the Bangladesh Survey Data

With the Bangladesh Bureau of Statistics (BBS), the Institute of Statistical Research, Bangladesh,
and the UN Statistical Office, UNICEF’s Bangladesh office helped develop and implement a
Multiple-Indicator Survey that helps to monitor many of the Mid-Decade and Year 2000 Goals. Seen
as a valuable complement to other data collection activities and a tool for validating routine reports
from the health services, the methodology has been institutionalised for continued use by the BBS.
Two surveys, in 1993 and 1994, have already been done and will be repeated annually by the BBS.
From the very beginning of the survey planning process, special emphasis was given to the
use of the survey results. This meant that at the very start, selection of the questions to be asked in
the survey was tied very precisely to the uses that would be made of the answers.
In Bangladesh the results of the Multiple-Indicator Surveys have served a myriad of
purposes, making full use of the time and effort which went into planning and carrying out these
surveys. In the following sections, some of the key areas in which the data have been put to action
REPORTING AND USING THE SURVEY FINDINGS 8.9

are described. Goal-monitoring has become a self-sustaining process in Bangladesh, with the
government initiating and directing these simple surveys. Not only have UNICEF programmes
benefited from the survey data, but so have
many other programmes supported by other
The surveys promote improved national
agencies. This diverse and widespread use has capacity for assessing the situation of
been possible because the Bangladesh office children and tracking progress toward
made certain to work with national partners to national goals for children.
build capacity and to see that results were
given wide exposure. Policy reports made the information accessible to other organizations in
readable and usable form. It is clear that the methodology and the results of the survey are fully
integrated into the social development schema of Bangladesh.

The Final Report: Monitoring the Goals

The survey results are used to report on the progress made toward achieving goals in the country.
They are disseminated in the book Progothir Pathey, meaning "the path of progress," which is
published annually by the BBS and is widely distributed to all ministries, district officials, NGOs,
donors and academic institutions. It contains national trend information on the mortality, fertility and
literacy indicators; maps showing the spatial coverage variations of selected indicators; summary
tables of data disaggregated by gender, area of residence (urban/rural, slum/non-slum) and ethnicity;
and district-specific tables reflecting the achievement of various indicators by gender. Progothir
Pathey has already become a reference publication for researchers, social scientists and development
workers in Bangladesh. The results of the survey are also being used to validate the completeness
of the routine reporting systems of the government.

Other Reporting Formats: Advocacy and Policy Dialogue

The results of the survey provide valuable information for initiating advocacy and policy dialogue,
not only for UNICEF, but for other UN and donor agencies and NGOs. The Center for Policy
Dialogue, an NGO, is currently carrying out an in-depth analysis of survey data to initiate a national
policy dialogue of some of the key disparity issues. The employment of maps to show the use of
iodized salt in the country has been critical in shaping government policy for importing iodized salt
and enforcing the universal salt iodization legislation in the country. UNICEF is also using these
results to publish their annual statistical card series&a set of pocket cards containing key data, their
implications and desired actions. These statistical cards are very popular amongst political leaders,
administrators and the donor community.
8.10 MULTIPLE-INDICATOR SURVEY HANDBOOK

Using the Survey Results for Expanded Action

The results of the survey are compared with the Mid- and Year 2000 Goals to calculate a "challenge
factor," which is the gap between the goal and the existing level of coverage, expressed as a per cent.
The challenge factor is then used to classify districts by their level of performance and therefore by
the level of attention and inputs necessary to bring them up to the level of other districts. The annual
average rate of change (AARC) is then used to set district-specific targets for various indicators. The
survey data are further analysed to calculate the design effect for each indicator in each district. This
design effect indicator is used to identify areas with the greatest intra-district variations in levels of
coverage. Based on this analysis, key district-specific strategies for action are identified. All this
information is presented in a situation analysis published and prefaced by the Planning Commission
in Bangladesh. KAMONA ("desire"), as it is called, is distributed to all ministries, district-level staff,
NGOs and donor agencies. A Bangla version of KAMONA is distributed as a part of the State of the
World’s Children report kit. It provides a local-level agenda for action for social development
workers in Bangladesh.

Subnational Planning: Looking Beyond National Averages

Data disaggregated by gender and area of residence have brought the need for subnational planning
to the forefront. With the help of the survey
results, it has been possible to identify the The survey should encourage policy
districts which are lagging behind in various makers to look at the information behind
sectors. This information has resulted in the the national averages.
development of more efficient and need-based
plans of action in particular areas for the achievement of the Mid-Decade and Year 2000 Goals.

Interagency Cooperation
The survey results have provided another opportunity for interagency cooperation. The availability
of survey data and its use for synthetic estimation of mortality and fertility at the district level now
allows UNICEF and UNDP to work together
to develop district-specific human develop-
The surveys provide special opportunities
ment indices (HDIs) for inclusion in the Ban- for promoting interagency cooperation
gladesh Human Development Report, 1995. between international organizations and
The Asia Development Bank (ADB) and the the host government.
World Bank have also asked UNICEF for
support in organizing special surveys, using the Multiple-Indicator Survey methodology, for
collecting subnational data necessary for project design. This cooperation opens new possibilities
REPORTING AND USING THE SURVEY FINDINGS 8.11

for UNICEF’s participation in the design of large projects and as well as in policy dialogue.

UNICEF Country Programme of Cooperation: Increased Efficiency and Effectiveness

The UNICEF Country Programme of Cooperation 1996%2000 uses the Convention on the Rights
of the Child as its framework. With most of the Mid-Decade Goals and some Year 2000 Goals
already achieved in Bangladesh, the new
programme places emphasis on equity and
reduction of disparities. The survey data have Survey results can help UNICEF make sure
that its own programming efforts and its
successfully identified interdistrict, interdivi- assistance are more efficient and effective.
sion, gender and ethnic variations and form the
basis for the design and development of
programmes for 1996%2000. The phasing plan of the Accelerated District Approach (ADA) of the
new proposed country programme uses challenge factors to identify districts requiring the most
attention and input. Thus, districts with the highest challenge factor&the weakest districts&are given
more time, input and attention to help them to catch up with the other districts in levels of coverage
and performance.

Staff Assignment and Budget Exercise

The staff assignment and relocation for the 1996/97 budget exercise is also based on the results of
the Multiple-Indicator Survey. For total office mobilisation to make an impact in achieving the goals,
senior and more experienced staff have been assigned to districts with high challenge factors.
Because it recognises the complexity of working in a high%challenge factor district, UNICEF
Bangladesh has limited the responsibilities of the staff in those areas to facilitating the achievement
of the goals in only one district, while staff assigned to low%challenge factor districts might be
covering more than one district. This approach has allowed UNICEF Bangladesh to more rationally
reallocate its staff (and even retrench some posts) without affecting overall office performance and
efficiency.

Reviews of Progress by Heads of State, Mayors and the Public

In country after country attention at the highest level of policy-making to the needs of children and
women has turned goal-setting into visible improvements. There is a demand for current, reliable
information to assess problems and progress. The time-bound, measureable nature of the Mid-
Decade and Year 2000 Goals has set a premium on responding to the demand to know.
In the past, surveys or other monitoring systems all too often meant a lengthy, costly process
where the results came long after policy makers had posed the questions. The desire to know, and
8.12 MULTIPLE-INDICATOR SURVEY HANDBOOK

the potential for action, had then passed. Final reports resembled academic treatises, discouraging
even those dedicated to learning from the results.
Today, high-level policy concern, skilled summaries by planners and the media as well as
action by programme managers are making it possible for the public to demand and learn the
progress being made toward goals.

O EXAMPLES:
The president of Mexico has presided over six public meetings to evaluate progress on the
National Programme of Action.
Mayors in the Philippines in the 60-member League of Cities initiated annual reports of
progress to track the impact of City Plans of Action for children to achieve the goals.1

This handbook provides you with the means to work with others to produce current assessments of
progress in a focused, rapid, cost-effective fashion. We hope you will find it a useful resource. Each
survey provides an opportunity to raise awareness, foster dialogue between health workers,
community leaders and public officials, and to take corrective action. Measuring and presenting
current status on the indicators is a most powerful driving force toward informed action for children.

1
For a summary of national and subnational action and status in the Philippines, see Progress of the
Philippines: 1995 Mid-Decade Goals (Manila: Government of the Republic of the Philippines and UNICEF,
1994).
REPORTING AND USING THE SURVEY FINDINGS 8.13

Box 8.3
Checklist for Reporting Your Results

3 Produce a preliminary report. Include:

U aims and objectives of the survey
U major results
U discussion and preliminary recommendations
U sample and methodology
U field work: questions asked, how they were asked, and who asked them
3 Produce an executive summary. Remember to:
U explain why mid-decade goal indicators are measured
U be sure of your findings
U include more descriptive and comparative information than technical
information
U summarise progress in achieving goals, according to indicators measured by
the survey
U give definitions of each indicator
U describe the survey steps
U provide the country trends for goal indicators that have past data available for
comparison
U present the main recommendations
3 Produce a full technical report. Include:
U an executive summary
U aims and objectives of the survey
U details of the sampling techniques used
U a copy of the questionnaire used
U details of the field work
U data-entry procedures
U an evaluation of data quality
U the results
U an interpretation of the results, comparing them with results from other
sources, and with data from neighboring countries
U conclusions, recommendations and acknowledgements
3 Make the reports of survey results fully accessible to all organizations which
might make use of them. Ensure the widest possible dissemination of results by
calling discussion meetings with donors, ministry officials and community leaders,
and by ensuring press coverage of these meetings.