PRIVACY PROGRAM MANAGEMENT

ONLINE TRAINING TRANSCRIPT

MODULE 7: PRIVACY OPERATIONAL LIFE CYCLE—
SUSTAIN: MONITORING AND AUDITING PROGRAM PERFORMANCE

Introduction

Module 7 introduction

Making informed choices and decisions on applying privacy-enhancing technology, process improvements
and performance measurements has become a complicated challenge for privacy managers.

Metrics serve as key performance indicators that can be used to set and attain business goals and
objectives. This module will help you recognize the primary, secondary and tertiary audiences for your
privacy program analysis. It will also summarize four types of metric analysis and identify the phases of a
privacy program audit.

Selecting and analyzing metrics

Learning objectives

• Recognize the primary, secondary and tertiary audiences for your organization’s privacy program
analysis

• Review the responsibilities of a metric owner

• Summarize four types of metric analysis

Developing metrics

A metric is a unit of measurement that should be as objective as possible and provide data that helps to
answer specific questions about business operations.

The statement, “you can never have enough,” does not apply to using metrics. Since data collection,
storage and analysis are expensive functions, you will not want metrics that provide no value. An
organization should focus on developing generic privacy metrics to reflect data privacy compliance, data-
driven decision-making, and the overall impact of the privacy program.

For which areas will your organization need to develop metrics? Select from the categories shown here,
then click “Submit.”

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 2

Collection
Responses to data subject inquiries
Use
Retention
Disclosure to third parties
Incidents
Privacy impact assessments/data protection impact assessments
Privacy risk indicators
Employee training
Percent of functions represented by governance mechanisms

There are no right or wrong answers here. Each organization will need to decide which privacy metrics are
most critical to evaluating program effectiveness.

Identifying the intended audience

As a first step in selecting relevant metrics, identify the intended metric audience—the relevant
stakeholders who will use the data to view, discuss and possibly make organizational strategic decisions.

Click the boxes to view typical members of each audience tier. These will vary by organization.

Primary audience

Who typically makes up the primary audience?

Legal and privacy officers, senior leadership, chief information officer (CIO), program managers
(PM), information system owners, chief information security officer (CISO) and chief privacy
officers (CPO).

Secondary audience

Who typically makes up the secondary audience?

Chief financial officer (CFO), training organizations, human resources (HR), inspectors general (IG),
HIPAA security officials

Tertiary audience

Who typically makes up the tertiary audience?

External watchdog groups, sponsors, stockholders

What differentiates these audiences?

The level of interest, influence, ownership and responsibility of privacy within the business
objectives. (For example, within a U.S. healthcare organization, a metrics audience may include a
HIPAA privacy officer, medical interdisciplinary readiness team (MIRT), senior executive staff and
covered entity workforce.)

Metric owner role

A metric owner is a process owner, champion and advocate responsible for management of the metric
throughout the metric life cycle. This person should have privacy knowledge, training and experience to
limit possible errors interpreting privacy-related laws, regulations and practices.

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 3

Review the metric owner responsibilities shown here.

An effective metric owner must:

• Know what is critical about the metric: Ask why the output is important and how the metric fits into
the business objectives.
• Monitor process performance with the metric.
• Keep process documentation up to date to ensure all audiences have a clear definition of the metric
and how it should be used. In addition:
o Minimize variance within a metric
o Develop documentation of metrics using flowcharts, visual displays, graphics and other
methods
o Champion the metric in meetings, working groups and in other organization communications
• Perform regular reviews. Determine if the metric is still required, capable to meet goals, and
provides value to the organization.
• Ensure improvements are incorporated and maintained in the process.

Analyzing metrics

Once metrics are created and data collected, a data analysis is conducted. Where possible, the privacy
professional should consider the use of automated tools or methods to gather, sort and report data.

Four common ways to analyze privacy program metrics are trend analysis, return on investment (or ROI),
business resiliency and privacy program maturity. Click to explore each.

Trend analysis

Ensures data is interpreted correctly and apparent relationships are meaningful and significant

Types:
1. Time series: Shows trends in an upward or downward tendency. Example: Number of
privacy breaches over time.
2. Cyclical component: Shows weekly, monthly or yearly data describing any regular
fluctuations. Example: Measuring the number of privacy breaches in the month after an
organization rolls out new privacy training—and then every three months to see if the
number steadily increases as distance from training increases.
3. Irregular component: Also known as “noise”—this is what is left over when the other
components of the series (time and cyclical) have been accounted for and is the most
difficult to detect. Example: The absence or indication of privacy breaches.

Return on investment

Return on investment is an indicator used to measure the financial gain or loss (value) of a project
in relation to its cost. ROI = (Benefits – Costs)/Costs. Privacy ROI helps provide justification to pay
for a good privacy program by defining metrics to measure the effectiveness of investments and
the cost to protect personal data. Click each asset type to learn more.

1. Personnel assets (users): Tracking measures that aim to reduce the chance of accidental or
intentional action by users either inside the organization, like employees and business
partners, or outside the organization, like hackers and bad guys. Their actions can alter,
destroy, misappropriate, misuse, misconfigure, distribute or make unavailable an
organization’s assets and data.
2. Information technology (IT) assets: Implementation and monitoring of hardware and
software assets with technical features that collectively protect the organizational assets
and data, achieving and sustaining confidentiality, integrity, availability and accountability.

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 4

3. Operational management assets: Creating and administering policies and standard

operating procedures that define the interaction between users, systems and system
resources.

Business resiliency

• According to IBM, business resiliency is “the ability to rapidly adapt and respond to business
disruptions and to maintain continuous business operations, be a more trusted partner, and enable
growth.”
• Metrics are associated with data privacy, system outages and other factors, as defined by the
business case and organization objectives.
• The business continuity or disaster recovery office (if it exists) should be contacted to assist in the
selection and use of data for this metric type.

Privacy program maturity

• You can select from a few Privacy Maturity Models (PMM) or develop a custom one for your
organization. Regardless of the PMM you use, it should define how to determine the maturity level
of your privacy programs and operations. Maturity is a useful metric because it focuses on a scale
as opposed to an endpoint. For example, acceptable data privacy protections may be in place
without being the “most mature.” Most maturity models use five maturity levels. These are the
levels as defined by the AICPA/CICA Privacy Maturity Model:
o Level 1 (Ad hoc): Informal, incomplete, undocumented and undefined
o Level 2 (Repeatable): There is structure and consistent focus on improvement
o Level 3 (Defined): Defined and documented with consistency
o Level 4 (Managed): Requirements and controls are in place with metrics
o Level 5 (Optimized): Deliberate and continuous process improvement
• Once the baseline assessment has been established, the organization can decide at which level of
maturity it ultimately wants or needs to operate. Ideal maturity levels can be challenging to
pinpoint and not all components of a program need to be at the same level.

ROI: Determining value

ROI analysis provides the quantitative measurement for the costs, benefits, strengths and weaknesses of
the organization’s privacy controls. Its goal is to maximize the benefits of investments that generally do
not generate revenue; rather, they prevent loss. To conduct an ROI analysis, you must define the value of
an asset.

What should an organization consider when determining the value of information assets? Click the
checkmarks below to reveal possible responses.

✓ Cost of producing the information asset

✓ Value of the asset on the open market
✓ Cost of reproducing the asset if it is lost, damaged or destroyed
✓ The asset’s benefit in meeting the organization’s mission and goals
✓ Repercussions to the organization if the asset is not readily available
✓ Cost to the organization if the asset is subject to unauthorized release, destruction or alteration
✓ Loss of public confidence in the organization if the asset is not handled correctly
✓ Loss of credibility and embarrassment to the organization if the asset’s security is compromised

Metrics: Reporting

Metrics are vital to the privacy program. DPOs and other privacy leaders must report to the board of
directors or senior leadership on privacy matters, and metrics can demonstrate compliance. Click the link
for a template of a DPO report to management, which identifies several categories of metrics, including:

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 5

• Defending the company’s systems and data

• Complying with legal responsibilities and regulations
• Advising the business

[Link]

From an expert: Metrics

Aaron Weller, CIPP/US, CIPM, CIPT, FIP, President and Co-founder, Ethos Privacy

When I’m looking at, what should privacy programs measure, really, for me, it’s thinking about, if it’s very
easy to measure, it’s probably not going to tell you very much; it’s not very insightful. But if it is
insightful, it’s not going to be very easy to measure. And I think that’s where a lot of people get caught
up. They measure the things that are easy to measure, and they report those things out. And they’re not
necessarily telling them very much, or they’re not telling them what they think they are. For me, when I’m
looking at, what do you measure, the real thing that I’m comparing every metric against is, what is it
actually, if it changes, what do I do about it? Because if a metric changes and it won’t change my
behavior, it’s probably a bad metric.

Let’s look at privacy impact assessments, right? There’s something that’s fairly common. People do them
all the time. Usually I see them measured, “How many did we get done this month?” Now, “How many did
we get done this month?” is generally—it doesn’t tell me very much. What if I do five or six in a month,
but there are 500 new things going into production? What if I do five or six a month, but my security team
is actually doing security reviews of 200 a month? The five or six by itself, even if it’s better than what we
did last month, doesn’t really tell you very much in itself.

So, the way that I like to think about metrics, which should help you within your own program, start from
thinking about: What do the board or executive management really care about? For example, if they say,
“Are we going to have any significant privacy risks that are going to be caused by, we make mistakes and
it goes into things that are in production?” People can see them outside of the organization. That’s very
hard to measure directly. But if you think about all of the activities that you could do to help prevent that.

So, thinking about, do we do an enterprise privacy risk assessment every year? Do we have a good sense
of kind of what those macro risks are? Do we have our privacy impact assessment process or privacy by
design built out across the organization? Are we doing reviews of everything that’s going to be high risk?
Do we think that’s important? There’s a few of those different things that you could see build up together
to help answer that executive question. And that for me is really the key to metrics. It’s thinking about,
how would I use this to change behavior? And then, how do I answer some of those harder questions that
it’s very hard to get from one number?

Another thing to think about with metrics is looking at leading vs. lacking indicators. So, again, if I go
back to my privacy impact assessment example, let’s say that we had a trend where we can see we’re
actually doing about 30 or 40% more privacy impact assessments every month. Right? You could say,
“That’s actually really good. I’m going in a good direction.” But at some point, you’re going to exceed the
capacity of the team that’s doing that. You’re either going to need to make the process more efficient or
you’re going to need to add additional resources to be able to deliver all of those things in a timely
manner. Now, if you do have this trend, the missing piece of information that you need to work out is,
where’s that limit? Where am I going to run out of resources? So, thinking about those limits before you
get to them and break them, you don’t want the first time you know about it to be, oh, we’re actually
seeing it now. The time it takes to respond to all of these tickets has gone up from five days to fifteen
days and now everyone’s yelling at us.

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 6

You can use metrics to get ahead of some of those things by looking out into the future and extrapolating
based on the data you have and knowing kind of some of those resource limitations of the team. So, I
would encourage you to think about, how do we use metrics to really achieve some of those broader
outcomes? How do I, instead of just reporting that something broke, think about, how do I identify these
things before they break?

And the other important thing, is to really think about, if it’s really easy to measure, what value are you
getting out of it? I’ve seen people produce these very beautiful dashboards. And when you say, “Well,
okay, how did you use this dashboard to actually manage your program?” They don’t know how to answer
that question. So, if you’re looking at metrics and you don’t have a good way of saying, “I’m using these
metrics to manage and improve the program,” or “to change resources around between different things,”
or “to take some other kind of action,” you probably need to go back and say, “Why am I capturing the
metrics at all? Are there other things that I could do that would actually be more useful with the same
amount of time and effort?”

So, one of the interesting things with metrics as well is that we’re starting to see them actually being
mandated by law. So, from July the 1st, 2021, if you have more than, I think it’s half million California
consumers, you’re required to actually publish metrics around the data subject rights that you’ve
processed in the previous year. So, the number, the average time to respond. And what’s really
interesting now is we actually have the ability to compare across different privacy programs with one very
specific piece.

And it’s interesting if you go and look at this. And there’s been some research that’s been published
around doing comparisons between different companies. Some companies have a huge number of access
requests, whereas others have very, very few. Some have a large number of do-not-sell requests,
whereas others, even in the same industry, have vastly different statistics and then the time that it takes
to respond. And this is really interesting when you’re looking to compare; if you’ve got two companies that
are very similar. And one turns around these requests in a day on average, and the other one takes 28 or
30 days, you kind of get a good sense of how that process is working underneath. So, I think this is a
start of a trend that’s going to be pretty interesting to see. You know we’ve seen this in financial reporting
and a lot of other areas where organizations are required to kind of standardize some of the things they
do so that people have a really good sense of, you know, should I work with this organization or that
organization? And I think it’s interesting for us in the privacy profession to think ahead to, what if there
were metrics that I had to report across all of my privacy program, not just this little piece that we do
right now?

So, it’s an interesting area, that I think, you know, we want to get ahead of as privacy professionals and
think about what are the things that we would want to actually present and measure about our program?
And what would we do if we were actually required by law to present some of these metrics? But it causes
a real problem, because some of those things where they’re not externally visible about how the program
works, I think we’re starting to kind of lift up that curtain of transparency. And it’s really interesting when
you get some additional insights into how the internal workings of privacy programs work, which I think
we really haven’t had today.

One of the other things to think about with metrics is that not every metric is appropriate for every
audience. If you’ve ever tried to present to an executive, you’ll know that they like things in three bullet
points or less. And sometimes, a lot of the metrics that we try to produce to run our programs, the
operational metrics, can be very down in the weeds. So, it’s important when you’re thinking about
capturing metrics, who the audience is for these metrics and what is the story that I need to tell around
them.

Because the raw numbers themselves, you know that there’s lies, damn lies and statistics. And even
having the same data set, two different people might be able to tell two completely different stories

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 7

around it. One of which will be effective in driving the outcome that they’re looking for. So, I think it’s
really important when you do start to gather some data, as you’re building a metrics program, look at
them with a critical eye and say, “What is the story that this is telling me? Would I be able to convince
somebody else of the same story? Are there multiple interpretations of this data?” And then, “How would I
present this?” And I look at this in three different ways. So, one would be from a management
perspective. The person that’s actually running that part of my program—how would I use this data to
help that person improve? Either become more effective or more efficient.

Another piece would be, let’s say that I’m talking about training, and I need to go and influence people
outside of the privacy program to get their teams to take the privacy training that we’ve created. So,
you’ve then got to go … and maybe that’s a comparability metric across. Here are all of the different
divisions in the company and this is the percentage of people that have actually completed the training,
right? “You, Mr. Executive, Ms. Executive, are lagging behind your peers in this.” So that could be based
on the similar data, but we’re presenting it in a different way with a different story.

And then from an executive perspective, maybe rolling all of that up and saying, “Look, we’re actually at
70 percent completion for the year. We do have a couple of areas where they’re at 40 percent but we’re
taking action with those groups directly to engage and to bring these percentages up. And the next time
we chat I can tell you if that was successful.” Because then you’ve got that top-down pressure as well.

So, you’re using some of those same underlying numbers to tell a few different stories and to try and
achieve some different objectives. From getting the people on the ground to complete the training, down
to really, then, having that influence that it should be something that’s important. So, I think that’s
something else to think about when you’re thinking about any kind of metric. Who is the audience? What’s
the story I’m trying to tell? And then, how can I see whether that behavior has actually changed that I’m
trying to influence? Because metrics should be used to try and influence behavior, to improve things. Not
just to exist for their own benefit.

Summary

• A metric provides data that helps to answer specific questions about business operations. An
organization should develop generic privacy metrics to reflect data privacy compliance, data-
driven decisions and the overall impact of the privacy program.
• Metrics have primary, secondary and tertiary audiences. Differences between the audiences are
based on interest level, influence, ownership and responsibility of privacy within the business
objectives.
• Typical members of a primary audience include the legal and privacy officers, senior leadership,
CIOs, CSOs, PMs, information system owners and CISOs. Secondary audience members include the
CFO, training organizations, HR, IGs and HIPAA security officials. The tertiary audience includes
external watchdog groups, sponsors and stockholders.
• A metric owner is responsible for managing the metric throughout its life cycle. Responsibilities
include knowing what is critical about the metric and how it fits into business objectives; monitoring
performance with the metric; updating process documentation (including the metric’s definition);
performing regular reviews; and incorporating improvements into the process.
• Four common ways to analyze privacy program metrics are trend analysis, return on investment
(or ROI), business resiliency and program maturity.
• ROI analysis provides quantitative measurement for the costs, benefits, strengths and weaknesses of
an organization’s privacy controls in order to maximize the benefits of investments that prevent loss.

Monitoring

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 8

Learning objectives

• Explore various types of privacy program performance monitoring and examples of each

• Review different forms of privacy program performance monitoring

Types of monitoring

Without a formal process to monitor privacy requirements, the organization cannot be reasonably assured
that personal information is handled appropriately and aligned to the organization, compliance
expectations and policy requirements.

You can monitor to track compliance and risk, organizational alignment with regulatory and legislative
changes, and vulnerabilities in the internal and external environments.

Note that regulatory and environment monitoring both feed into compliance and risk monitoring.

Drag the example of monitoring to the appropriate category.

Compliance and risk

Review the collection, use and retention of personal information throughout the information life
cycle

Regulatory and legislative changes

Track using publications and/or external vendors

Environmental vulnerabilities

Monitor internal and external threats, including building access, data access and authentication,
and lack of awareness or training

Many forms of monitoring

There are many options and formats for auditing privacy program performance, including active scanning
tools, formal audits, dashboards and complaint tracking.

Click the boxes to view more examples of monitoring.

• Active scanning tools, such as data loss prevention (DLP) network, to identify risks to personal
information and monitor for compliance
• Audit activities, such as internal and external reviews of people, processes, technology and
financials
• Breach management practices, including breach monitoring: Driven by laws and regulations;
tracking breach type, severity, and time to remediation are especially important types of
monitoring
• Complaint monitoring: A formal process will track, report, document and provide resolutions to
complaints; protect the organization legally; and provide repeatable processes and tracking
mechanisms to ensure transparency and accountability. Details about the type and location of
complaints can provide early indicators of the potential for regulatory activity.

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 9

• Data retention/records management strategies: Should meet legal and business needs for
privacy, security and data archiving; monitor for potential areas for risk in retention schedules
or practices like excessive collection or inadequate controls
• Dashboards: Automated means for organizations to identify, document and manage their
existing risks and controls
• Control-based monitoring: Assesses the design and operational effectiveness of controls. Some
governance, risk and compliance (GRC) tools provide automated means to undertake some or
all of these checks.
• HR practice monitoring: E.g., hiring and termination; monitoring data; and monitoring building
access/use. HR is responsible for ensuring the privacy of employee personal information; some
kinds of workplace monitoring require additional privacy considerations.
• Monitoring internal and external conditions: Risks that exist because of changes in the
environment or changes to the industry; internal shifts such as mergers, acquisitions and
divestitures
• Regulation-based monitoring: For compliance with regulations and requirements
Suppliers/third parties: Supplier monitoring should include appropriate privacy and security
requirements, as well as provider performance, to ensure compliance to contract specifications,
laws, and policies

Continuous monitoring of program performance

Continuous monitoring—including audits and assessments, metrics, and frameworks—can provide the data
to help an organization ensure it is achieving its program goals.

Your continuous monitoring efforts should enable you to answer “yes” to the questions shown here.

Are you…

• Protecting personal information?

• Following policies, procedures and programs?
• Minimizing consequences via early detection and remediation?
• Providing feedback?
• Demonstrating your commitment to privacy management?

Summary

• Organizations can monitor privacy programs to track compliance and risk, organizational alignment
with regulatory and legislative changes, and vulnerabilities in the internal and external environments.
• Tracking compliance and risk involves reviewing the collection, use and retention of personal
information throughout its life cycle.
• Tracking regulatory and legislative changes is often done using publications and/or external
vendors.
• Tracking environmental vulnerabilities involves monitoring internal and external threats, including
building access, data access and authentication, and lack of awareness or training.
• Forms of privacy program performance monitoring include:
o Active scanning tools, such as data loss prevention (DLP) network
o Audit activities
o Breach monitoring, detection and notification
o Complaint monitoring
o Data retention/records management strategies
o Dashboards
o Control-based monitoring

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 10

o HR practices, such as hiring and termination; monitoring data; and monitoring building
access/use
o Monitoring internal and external conditions
o Regulation-based monitoring

Auditing

Learning objectives

• Review the definition of a privacy audit

• Identify five phases of auditing a privacy program

• Compare the three types of privacy program audits

Privacy audit definition

Review the definition of a privacy audit shown here, then click “Next.”

“Audits are an ongoing process of evaluating the effectiveness of controls throughout the organization’s
operations, systems, and processes … The purpose of a privacy audit is to determine the degree to which
technology, processes, and people comply with privacy policies and practices.” – Privacy Program
Management, Third Edition

Audit sustains the organization through monitoring and measuring privacy practices to comply with laws,
regulations, consent orders and industry practices.

An audit is different from an assessment, as it is more evidence-based. Assessments are generally less
formal and more anecdotal.

Privacy audits answer the following questions:

• Do the privacy operations do what they were designed to do?

• Are data privacy controls correctly managed?

Why audit?

What are some reasons to perform privacy audits? Select all that could apply to your organization, then
click “Submit.”

• Identified vulnerabilities and weaknesses that indicate risk

• A security or privacy incident
• Deterioration of a business function
• Indications of an insider threat
• Staffing, cutbacks and changes to priorities
• New subcontractors or third parties
• Unusual changes, such as higher numbers of privacy breaches, complaints or incidents
• New portfolio or industry base

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 11

Each item listed here is a potential reason to conduct a privacy audit. Audits should be conducted
regularly, regardless of whether a change or incident has occurred.

Auditing privacy (1)

Can you identify the five phases of a privacy program audit?

• Audit planning, audit preparation, audit, report and follow-up

• Audit planning, audit, analyze, report and follow-up
• Audit, audit analysis, audit review, report and evaluate
• Audit planning, audit preparation, audit, follow-up and renewal

For the purposes of this program, audit planning, audit preparation, audit, report and follow-up are the
five phases of a privacy program audit.

Be sure to maintain a comprehensive audit trail—a critical component of privacy monitoring—to

demonstrate your organization meets regulatory requirements and industry best practices.

Auditing privacy (2)

The high-level, five phase audit approach includes:

1. Audit planning
2. Audit preparation
3. The audit itself
4. Reporting
5. Follow-up

Click each number to learn more.

1. Audit planning: Risk assessment, schedule, selecting auditor, pre-audit questionnaire,

preparatory meeting/visit and checklist

2. Audit preparation: Confirm schedule; confirm and prepare checklists, sampling criteria and audit
plan

3. Audit: Meeting and audit execution

4. Report: Noncompliance records and categories (major/minor), audit report, closing meeting and
distribution

5. Follow-up: Confirm scope, schedule, methodology and closure

From an expert: The audit process

Antonis Patrikios, CIPP/E, CIPM, FIP, Partner, Dentons, IAPP faculty

Only one point that I want to make in relation to audits, just one. The rest is common sense. You work
with the relevant function, you … you prepare, you communicate. It’s a project that you need to handle,
and you need to handle well. There’s only one point I want to make about audits, and it sits somewhere
between these two. So, what happens if you get a report that tells you, you have a problem there? It

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 12

makes a recommendation. But what do you absolutely have to do after that? You need to fix the problem.
You need to fix…

So, you’ve gone through the process of audit to check yourself, and you’ve found that the server we were
talking about earlier was not patched, or the nurses are downloading from wherever. At that point, when
the report is revealed, there is corporate knowledge of incompliance in a part of the business, and then if
you don’t act on it to fix it, things could be really, really bad if something goes wrong.

First-party audits

Privacy program audits typically fall into one of three categories: first party, second party or third party.

A first-party audit acts as a self-assessment to evaluate the organization’s risk management culture;
identify privacy risk factors within systems, processes and procedures; and evaluate control design and
implementation to ensure proper risk management.

Read more

A first-party audit is performed by internal employees. Functions to review are determined by manpower
and compliance factors.

Self-certification does not exempt an organization from fulfilling obligations under applicable laws or
regulations.

An internal auditor may develop an audit work plan that will:

• Identify the areas to be audited

• Notify those offices of the plans
• Hold meetings and reviews
• Provide all communications
• Draft reports and presentations
• Lead management communications
• Close all audit matters
• Formalize reports and final meetings
• Perform follow-ups

Second-party audits

Second-party audits are often known as “supplier audits,” because they typically involve the organization
auditing existing suppliers or subcontractors.

Read more about supplier audits under the GDPR:

When a controller (or a processor under the GDPR) outsources any activity, responsibility is not
"outsourced."

It is important that the entity outsourcing any processing audits the supplier to ensure the supplier can
carry out the processing to the organization’s requirements and meet the organization’s obligations under
the GDPR (especially in relation to security of the personal data).

Third-party audits

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 13

Third-party audits are required under consent decree or by a regulator.

They are conducted by independent outside sources, for example, the data protection commissioner,
government officials or via an independent external assessment by subcontractors.

Read more details and examples of third-party audits

May align to various frameworks, for example:

• ISO 19011 provisional standards through joint auditing of environmental management (ISO 14001)
and quality management (ISO 9001) systems
• NIST SP 800-53 Rev 5 (9/23/2020)
• AICPA GAPP

Provide:

• A formal record of what was audited and when

• Insight into areas that comply/do not comply
• Details to support the findings
• Suggested corrective actions, with possible target dates

Audit findings must be communicated to affiliated stakeholders in the organization, who will then consider:

• Risk level/degree of compliance

• Accountability for correction (action plan)
• Mitigation costs
• Approval of remediation process (or justification for disapproval)

External auditors

What are some advantages and disadvantages to using external auditors? Drag and drop potential
answers below.

Advantages:

• Identifying weaknesses of internal controls

• Lending credibility to internal audit program
• Providing a level of unbiased, expert recommendations
• May give investors, regulators and the public greater confidence

Disadvantages:

• Cost/budget
• Time/schedule
• Time it takes to learn about the organization
• Confidentiality concerns

Periodic review process

Monitoring—of any type—is not useful unless the organization takes the time to analyze the results.
Ensure you build triggers into your monitoring process that signal the privacy officer to step back and
evaluate the program.

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 14

At regular intervals, consider the questions listed here.

When does your governance structure need revamping?

What triggers a policy review?

How often do audits happen?

What in an audit triggers a follow-up action?

Summary

• Audit involves monitoring and measuring privacy practices to comply with laws, regulations,
consent orders and industry practices.
• Audits should answer two questions: 1) Do the privacy operations do what they were designed to
do? 2) Are data privacy controls correctly managed?
• Privacy program audits typically fall into one of three categories: first party, second party or third
party.
• First-party audits are performed by internal employees. They are self-assessments used to
evaluate risk management culture; identify privacy risk factors; and evaluate control design and
implementation.
• Second-party audits, often known as “supplier audits,” typically involve the organization auditing
existing suppliers or subcontractors.
• Third-party audits are required under consent decree or by a regulator. They are conducted by
independent outside sources. They provide a formal record of what was audited and when, insight
into areas that comply/do not comply, details to support findings and suggested corrective actions.

Quiz

1. External watchdog groups, sponsors and stockholders typically make up which audience for privacy
program metrics?

Primary

Secondary

Tertiary

None of the above

2. Which of the following is NOT typically a responsibility of a privacy metric owner?

Tracking the costs of analyzing data for the metric on an organization’s profit and loss statement

Ensuring improvements are incorporated and maintained in the process

Scheduling regular reviews to determine if the metric is still required, capable of meeting goals and
providing value to the organization

Understanding how the metric fits into the organization’s business objectives

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 15

3. A privacy officer is trying to make the case to his CFO to invest more of the budget into incident
prevention and preparedness. He wants to show that the likely financial gain of this investment is greater
than the direct costs to the organization. Which category of metrics would be most useful to him?

Business resiliency

Program maturity

Return on investment (ROI)

Trend analysis

4. True or false? A privacy audit should reveal whether the privacy operations do what they were designed
to do and whether privacy controls are correctly managed.

True

False

5. Employing dashboards, active scanning tools, and data retention and records management strategies
are all ways to do what?

Monitor privacy program performance

Ensure an organization minimizes the processing of personal data

Increase information security

Provide resolutions to privacy-related complaints

6. Which category of audits may align to an ISO standard, NIST special publication or other industry
framework?

First-party audits

Supplier audits

Third-party audits

Data protection commissioner audits

7. Which of the following is a valid reason for an organization to conduct a privacy audit?

It has expanded its industry base

It has made staffing cutbacks and shifted its business priorities

There has been a confirmed security incident

All of the above

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.
 16

8. The privacy officer for a corporation is analyzing trends on a series of privacy program metrics. She
notices a conspicuous absence of privacy incidents in the past two years and wants to include this in her
reporting. This information is known as a(n):

Time series

Uncertainty variable

Irregular component

Cyclical component

Closing slide

You have completed Module 7: Privacy Operational Life Cycle—Sustain: Monitoring and Auditing Program
Performance.

Quiz answers

1. Tertiary
2. Track the costs of analyzing data for the metric on an organization’s profit and loss statement
3. Return on investment (ROI)
4. True
5. Monitor privacy program performance
6. Third-party audits
7. All of the above
8. Irregular component

*Quiz questions are intended to help reinforce key topics covered in the module. They are not meant to
represent actual certification exam questions.

©2023, International Association of Privacy Professionals, Inc. (IAPP). Not for reproduction, distribution or republication.