Data Management Processes: The Data Factory
You can compare a well-functioning data management process to a conveyer belt
in a factory. The first part of the conveyer belt brings raw materials into the
factory: wood or metal, for example. As the conveyer belt continues through the
factory, the raw wood and metal parts are sorted, shaped, assembled and
eventually transformed into useful products: chairs, cars or bottles, for example.
In the same way, a data management process starts by bringing in raw
materials: data collected from the project. As the data travels through the rest of
the process it is organized, stored, analyzed and, finally, transformed into useful
products, such as reports and decisions.
In this course, we have talked a lot about the first step: collecting data. But what
about all of the other steps in the data management process? Let’s explore the
entire data management process.
While each organization will have its own process for managing data, most data
generally go through these steps:
Step 1: Data Collection
This is the step that we have already explored. Data from the project are
collected using a variety of data collection methods and tools.
Step 2: Data Entry and Collation
After data are collected, they are entered into a data system. Your system might
be a digital database, an Excel spreadsheet, or a filing system in the central
office. This is usually the moment when data are organized. For example, all of
the responses to a single question might be grouped together. This process of
grouping similar data together is known as collation. Collation will make data
analysis much simpler. If you are using a digital data collection tool, data entry
and collation may happen automatically.
Step 3: Data Analysis, Verification, and Storage
Once data are in the data system, they can be analyzed, verified and stored.
Data analysis is an extremely complex subject, and we will not have time to
discuss it properly in this course. Generally speaking, however, data analysis
means using data to answer questions and to reach conclusions. Many different
people will probably participate in data analysis. For example, your project team
will analyze some data to make monitoring decisions, while your evaluation team
will analyze other data when conducting evaluations.
Data also need to be regularly verified. This ensures that data are being
accurately collected.
Additionally, data need to be stored. Most data should be stored for the duration
of the entire project. After the project is completed, some of this data will be
stored for future use. This process is sometimes called archiving. You should be
thoughtful about which data are archived, and ensure that they are stored
securely.
Step 4: Data Use
Eventually, data are put to some use. Some of the ways that data are commonly
used include:
Creating reports
Communicating outcomes to the community
Making project management decisions
Helping to design future projects
This is just a partial list of the many ways that data can be used. It is important
to plan for data use from the beginning of your project. Many organizations
collect lots of data but do not use very much of it. That is why we’ll spend some
time today thinking about how your project will use data.
Most data will go through these steps, although not necessarily in this order. Your
data management process will probably be much more complex than the four-
step process that you have just read. In the next section, you will have a chance
to decide how your team will manage data.
A note on data management
As you may have noticed, many of the steps that we have just described are,
such as data analysis or data verification, are complex, challenging subjects that
probably deserve their own courses. Additionally, ensuring data security
throughout the process is an enormously important issue, and is difficult to do
well. It is beyond the scope of this course to give helpful advice on all of these
topics. However, we hope that this overview gives you an idea of the types of
tasks that your team will be responsible for completing.
As we continue through this module, you will begin making some decisions
related to these subjects. However, as you do this, keep in mind that these are
complex subjects that we have not had the time to fully explore.
Data Flow Maps
As we just mentioned, your data management process probably will not be quite
as simple as the 4-step process that we just described. It is more likely that
different types of data will enter your system in different ways and at different
times; that some steps will be repeated or taken out of order; or that additional
steps will be added in.
Data Flow Mapping is an exercise to help you visualize and plan your data
management process in all of its complexity. Take a look at the example below,
which was prepared for a project that delivers HIV testing, counseling and
treatment to orphans and vulnerable children (OVC).
This data flow map shows this organization’s complex data management
process. It contains a lot of information. Let’s explore what this data flow map
shows, starting at the bottom. As we look at the data flow map, ask yourself: how
is this data management process similar to your project’s data management
process? How is it different?
The orange boxes at the bottom of the page are the three data collection tools:
Participant Intake Forms, which record demographic and health
information about individual OVCs.
HIV Testing and Counseling Records, which record the initial tests and
counseling sessions with participants.
Antiretroviral Treatment Records, which record the types of medical
treatment that participants receive at the clinics.
There are two roles that are responsible for using these tools to collect data.
These roles are shown in green just above the data collection tools:
Community Health Workers (CHW): There are 40 CHWs on the project.
These people are responsible for working with families and community
agencies to bring participants into the project. Every day they enter their
data in the digital database.
Field Officers: There are two Field Officers at the clinic, each responsible
for a different type of service. Every week they collate their clinic’s
treatment data and enter it into the digital database.
Once data has been entered into the digital database, it is sent to the central
office, where it can be analyzed and used by the:
Project Manager: The Project Manager monitors the data regularly, and
uses the information to make decisions. Additionally, several times a year
the project manager prepares reports and sends them on to the Executive
Director.
Executive Director: The Executive Director reviews and comments on
reports. She sends reports on to the organization’s Board of Directors,
donors, and government agencies. Once a quarter she has feedback
meetings with local community leaders where she shares findings from the
reports.
As you can see, this organization’s data flow map illustrates the complex journey
that data take from start to finish. It visually shows important information about
the data management process without going into too much detail.