A Framework for Evaluating
Process/Transaction-Level Exceptions and Deficiencies
Version 1
October 28, 2004
This framework is intended to be used for process/transaction level exceptions and
deficiencies and may be updated in the future to include frameworks related to
other exceptions and deficiencies.
Table of Contents
Page
Introduction and Purpose 1
Guiding Principles 4
Terminology 8
Chart 1 – Evaluating Process/Transaction-Level
Control Exceptions Found in the Testing of Operating
Effectiveness 10
Chart 2 – Evaluating Process/Transaction-Level
Control Deficiencies 11
Introduction and Purpose
This paper outlines a suggested framework for evaluating manual and automated
process/transaction-level exceptions and deficiencies resulting from the evaluation of a
company’s internal control over financial reporting. This paper should be read in
conjunction with Auditing Standard No. 2, An Audit of Internal Control Over Financial
Reporting Performed in Conjunction With an Audit of Financial Statements (AS2),
especially the definitions in paragraphs 8 through 10, the section on evaluating
deficiencies in paragraphs 130 through 141, the examples of significant deficiencies and
material weaknesses in Appendix D, and the Background and Basis for Conclusions in
Appendix E. The framework is not a substitute for AS2 and other relevant professional
literature.
Issuers and auditors may find this framework useful. It is not intended to, and should not,
be utilized when evaluating control deficiencies in IT general controls and other types of
exceptions and deficiencies.
The framework was developed by representatives of the following nine firms:
BDO Seidman LLP
Crowe Chizek and Company LLC
Deloitte & Touche LLP
Ernst & Young LLP
Grant Thornton LLP
Harbinger PLC
KPMG LLP
McGladrey & Pullen LLP
PricewaterhouseCoopers LLP
In addition, William F. Messier, Jr., Professor, Georgia State University, also contributed
to the development of the framework.
This framework reflects their views on a framework consistent with their understanding
of AS2.
The framework represents a thought process that will require significant judgment. The
objective of the framework is to assist knowledgeable and experienced individuals in
evaluating deficiencies in a consistent manner. The mere mechanical application of this
framework will not, in and of itself, necessarily lead to an appropriate conclusion.
Because of the need to apply judgment and to consider and weigh quantitative and
qualitative factors, different individuals evaluating similar fact patterns may reach
different conclusions.
The framework recognizes the requirement in AS2 to consider likelihood and magnitude
in evaluating deficiencies. It also recognizes that AS2.136 states:
In evaluating the magnitude of the potential misstatement, the auditor should
recognize that the maximum amount that an account balance or total of
1
transactions can be overstated is generally the recorded amount. However, the
recorded amount is not a limitation on the amount of potential understatement.
The auditor also should recognize that the risk of misstatement might be different
for the maximum possible misstatement than for lesser possible amounts.
The framework applies these concepts through the evaluation of a combination of
magnitude and likelihood. Because of the wide variety of control types, population
characteristics, and test exception implications, the group did not undertake to develop a
purely quantitative model. Instead, the framework considers quantitative and qualitative
factors.
Other key assumptions considered in developing the framework include:
o The testing of controls generally relates to significant processes and major classes
of transactions for relevant financial statement assertions related to significant
accounts and disclosures. Therefore, the underlying assumption is that all
exceptions/deficiencies resulting from the testing must be evaluated because they
relate to accounts and disclosures that are material to the financial statements
taken as a whole.
o The purpose of tests of controls is to achieve a high level of assurance that the
controls are operating effectively. Therefore, the sample sizes used to test
controls should provide that level of comfort. In cases in which samples are
selected using a statistically based approach, sample sizes for frequently operating
manual controls that result in less than a 90% level of confidence that the upper
limit deviation rate does not exceed 10% typically would not provide a high level
of assurance. (Refer to the AICPA Audit and Accounting Guide, Audit Sampling).
o The magnitude of a control deficiency (i.e., deficiency, significant deficiency, or
material weakness) is evaluated based on the impact of known and/or potential
misstatements on annual and interim financial statements.
While some of the concepts discussed in this paper relate to statistical sampling, the
framework does not require the use of statistical sampling. A statistical sample is (1)
selected on a random or other basis that is representative of the population and (2)
evaluated statistically. In tests of internal controls, it may be impractical to select
samples randomly, but they should be selected in an unbiased manner.
This paper does not address the determination of materiality. Reference, in that regard,
should be made to AS2.23, which states:
The same conceptual definition of materiality that applies to financial reporting
applies to information on internal control over financial reporting, including the
relevance of both quantitative and qualitative considerations.*
o The quantitative considerations are essentially the same as in an audit of
financial statements and relate to whether misstatements that would not be
prevented or detected by internal control over financial reporting,
individually or collectively, have a quantitatively material effect on the
financial statements.
2
o The qualitative considerations apply to evaluating materiality with respect
to the financial statements and to additional factors that relate to the
perceived needs of reasonable persons who will rely on the information.
AS2.6 describes some qualitative considerations.
* AU sec. 312, Audit Risk and Materiality in Conducting an Audit, provides
additional explanation of materiality.
3
Guiding Principles
The principles set forth below correspond to the box numbers on the appropriate charts
included in this paper.
The evaluation of exceptions and deficiencies is an iterative process. Although this paper
depicts the evaluation process as a linear progression, it may be appropriate at any point
in the process to return to and reconsider any previous step based on new information.
Evaluating Process/Transaction-Level Exceptions
Found in the Testing of Operating Effectiveness (Chart 1)
Box 1. All exceptions should be evaluated quantitatively and qualitatively. A
thorough understanding of the cause of the exception is important in
evaluating whether a test exception represents a control deficiency. This
evaluation should consider the potential implications with regard to the
effectiveness of other controls, e.g., the company’s information technology
general controls (“ITGC”) and other COSO components.
In concluding whether the test objective was met, considerations include:
o The deviation rate in relation to the frequency of performance of the
control (e.g., absent extending the test, there is a presumption that an
exception in a control that operates less frequently than daily is a
control deficiency).
o Qualitative factors, including exceptions that are determined to be
systematic and recurring or that relate to the factors outlined in AS
2.133, 139 and 140.
o Whether the exception is known to have resulted in a financial
statement misstatement (e.g., there is a presumption that an exception
that results in a financial statement misstatement in excess of the level
of precision at which the control is designed to operate, is a control
deficiency).
A control objective may be achieved by a single control or a combination
of controls. A test of controls may be designed to test a single control that
alone achieves the control objective or a number of individual controls that
together achieve the control objective.
Box 2. If the test objective is not met, consideration should be given to whether
additional testing could support a conclusion that the deviation rate is not
representative of the total population. For example, if observed exceptions
result in a non-negligible deviation rate, then the test objective initially is
not met. In a test designed to allow for finding one or more deviations, the
test objective is not met if the actual number of deviations found exceeds
the number of deviations allowed for in the plan.
4
Box 3. If the test objective initially is not met, then there are two options:
o If the observed exceptions and resulting non-negligible deviation rate
are not believed to be representative of the population (e.g., because of
sampling error), the test may be extended and re-evaluated.
o If the observed exceptions and resulting non-negligible deviation rate
are believed to be representative of the population, the exceptions are
considered to be a control deficiency and its significance is assessed.
Evaluating Process/Transaction-Level Control Deficiencies (Chart 2)
Step 1. Determine whether a significant deficiency exists:
Box 1. When evaluating deficiencies, potential magnitude (inconsequential, more
than inconsequential, or material) is based on the potential effect on both
annual and interim financial statements. The potential magnitude of a
misstatement of annual or interim financial statements of not more than
inconsequential results in the deficient control being classified as only a
deficiency, absent any qualitative factors, including those in AS 2.9, 137,
139, and 140. Potential magnitude of misstatement may be based on gross
exposure, adjusted exposure, or other appropriate methods that consider the
likelihood of misstatement.
Box 2&3. If there are controls that effectively mitigate a control deficiency, it is
classified as only a deficiency, absent any qualitative factors, including
those in AS 2.9, 137, 139, and 140. Such controls include:
o Complementary or redundant controls that achieve the same control
objective
o Compensating controls that operate at a level of precision that would
result in the prevention or detection of a more than inconsequential
misstatement of annual or interim financial statements
Boxes 1, 2, and 3 should be considered separately. Adjusted exposure
should not be reduced by the quantitative impact of the compensating and
complementary or redundant controls.
Box 3. An unmitigated deficient control that results in a control objective not
being met related to a significant account or disclosure generally results in
a more than remote likelihood of a more than inconsequential misstatement
of annual or interim financial statements and, therefore, is at least a
significant deficiency.
5
Step 2. Determine whether a material weakness exists:
Box 4. The potential magnitude of a misstatement of annual or interim financial
statements that is less than material results in the deficient control being
classified as only a significant deficiency, absent any qualitative factors,
including those in AS 2.9, 137, 139 and 140. Potential magnitude may be
based on gross exposure, adjusted exposure, or other appropriate methods
that consider the likelihood of misstatement.
Box 5. Compensating controls that operate at a level of precision that would result
in the prevention or detection of a material misstatement of annual or
interim financial statements may support a conclusion that the deficiency is
not a material weakness.
Box 6. In evaluating likelihood and magnitude, related factors include but are not
limited to the following:
o The nature of the financial statement accounts, disclosures, and
assertions involved; for example, suspense accounts and related party
transactions involve greater risk.
o The susceptibility of the related assets or liability to loss or fraud; that
is, greater susceptibility increases risk.
o The subjectivity, complexity, or extent of judgment required to
determine the amount involved; that is, greater subjectivity,
complexity, or judgment, like that related to an accounting estimate,
increases risk.
o The cause and frequency of known or detected exceptions for the
operating effectiveness of a control; for example, a control with an
observed non-negligible deviation rate is a deficiency.
o The interaction or relationship with other controls; that is, the
interdependence or redundancy of the control.
o The possible future consequences of the deficiency.
o An indication of increased risk evidenced by a history of
misstatements, including misstatements identified in the current year.
o The adjusted exposure in relation to overall materiality.
This framework recognizes that in evaluating deficiencies, the risk of
misstatement might be different for the maximum possible misstatement
than for lesser possible amounts.
As a result of this additional evaluation, determine whether the likelihood
of a material misstatement to both the annual and interim financial
statements is remote. In extremely rare circumstances, this additional
evaluation could result in a judgment that the likelihood of a more than
inconsequential misstatement to both the annual and interim financial
statements is remote.
6
Box 7&8. When determining the classification of a deficiency, consider AS 2.137,
which states:
When evaluating the significance of a deficiency in internal control
over financial reporting, the auditor also should determine the level
of detail and degree of assurance that would satisfy prudent
officials in the conduct of their own affairs that they have
reasonable assurance that transactions are recorded as necessary to
permit the preparation of financial statements in conformity with
generally accepted accounting principles. If the auditor determines
that the deficiency would prevent prudent officials in the conduct
of their own affairs from concluding that they have reasonable
assurance,* then the auditor should deem the deficiency to be at
least a significant deficiency. Having determined in this manner
that a deficiency represents a significant deficiency, the auditor
must further evaluate the deficiency to determine whether
individually, or in combination with other deficiencies, the
deficiency is a material weakness.
Note: AS2.9 and .10 provide the definitions of significant
deficiency and material weakness, respectively.
* See SEC Staff Accounting Bulletin Topic 1M2, Immaterial
Misstatements That Are Intentional, for further discussion about
the level of detail and degree of assurance that would satisfy
prudent officials in the conduct of their own affairs.
Additional considerations related to misstatements identified:
A greater than de minimis misstatement of annual or interim financial statements
identified by management or by the auditor during a test of controls or during a
substantive test is ordinarily indicative of a deficiency in the design and/or operating
effectiveness of a control, which is evaluated as follows:
o The design and/or operating deficiency(ies) that did not prevent or detect
the misstatement should be identified and evaluated based on Chart 2 –
Evaluating Process/Transaction-Level Control Deficiencies – applying the
following:
A known or likely (including projected) misstatement that is
inconsequential to annual or interim financial statements is at least
a deficiency.
A known or likely (including projected) misstatement that is more
than inconsequential to annual or interim financial statements is a
strong indicator of a significant deficiency.
A known or likely (including projected) misstatement that is
material to annual or interim financial statements, as addressed in
AS2.140, is at least a significant deficiency and a strong indicator
of a material weakness.
o The implications on the effectiveness of other controls, particularly
compensating controls, also should be considered.
7
Terminology
Adjusted exposure – gross exposure (see below) multiplied by the upper limit deviation
rate.
Compensating controls – controls that operate at a level of precision that would result in
the prevention or detection of a misstatement that was more than inconsequential or
material, as applicable, to annual or interim financial statements. The level of precision
should be established considering the possibility of further undetected misstatements.
Complementary controls – controls that function together to achieve the same control
objective.
Control deficiency – a deficiency in the design or operation of a control that does not
allow management or employees, in the normal course of performing their assigned
functions, to prevent or detect misstatements on a timely basis.
o A deficiency in design exists when (a) a control necessary to meet the control objective
is missing or (b) an existing control is not properly designed so that, even if it operates
as designed, the control objective is not always met.
o A deficiency in operation exists when a properly designed control does not operate as
designed, or when the person performing the control does not possess the necessary
authority or qualifications to perform the control effectively.
Control objective – the objective(s) related to internal control over financial reporting to
achieve the assertions that underlie a company’s financial statements.
Gross exposure – a worst-case estimate of the magnitude of amounts or transactions
exposed to the deficiency with regard to annual or interim financial statements, without
regard to the upper limit deviation rate or likelihood of misstatement, and before
considering complementary, redundant, or compensating controls. Factors affecting gross
exposure include:
o The annual or interim financial statement amounts or total transactions exposed to the
deficiency.
o The volume of activity in the account balance or class of transactions exposed to the
deficiency that has occurred in the current annual or interim period or that is expected
in future periods.
Inconsequential
o Potential misstatements equal to or greater than 20% of overall annual or interim
financial statement materiality are presumed to be more than inconsequential.
o Potential misstatements less than 20% of overall annual or interim financial statement
materiality may be concluded to be more than inconsequential as a result of the
consideration of qualitative factors, as required by AS2.
Material weakness – a significant deficiency, or combination of significant deficiencies,
that results in more than a remote likelihood that a material misstatement of the annual or
interim financial statements will not be prevented or detected.
8
Potential misstatement – an estimate of the misstatement that could result from a
deficiency with a more than remote likelihood of occurrence.
Redundant controls – controls that achieve the same control objective.
Remote likelihood – the chance of the future event or events occurring is slight.
Significant deficiency – a control deficiency, or combination of control deficiencies, that
adversely affects the company's ability to initiate, authorize, record, process, or report
external financial data reliably in accordance with generally accepted accounting principles
such that there is more than a remote likelihood that a misstatement of the company's
annual or interim financial statements that is more than inconsequential will not be prevented or
detected.
Test objective – the design of the test of a control activity to determine whether the
control is operating as designed, giving consideration to:
o The frequency with which the control operates
o The desired level of assurance in combination with the reliability of the control, for
example, whether the control is designed to achieve the control objective alone or in
combination with other controls
o The number of exceptions expected
Upper limit deviation rate – the statistically derived estimate of the deviation rate based
on the sample results, for which there is a remote likelihood that the true deviation rate in
the population exceeds this rate (refer to AICPA Audit and Accounting Guide, Audit
Sampling).
9
CHART 1 – Evaluating Process/Transaction-Level Control
Exceptions Found in the Testing of Operating Effectiveness
Box 1. Examine and understand cause and results of
exceptions. Was the test objective met (e.g., was the
actual deviation rate less than or equal to the planned
deviation rate) ?
No
Box 2. Considering the results of management’s and
the auditor’s testing and the information obtained in Yes
Box 1, could additional testing support a conclusion
that the deviation rate is not representative of the total
population?
No
Yes
No Box 3. Extend testing and re-evaluate. Was the test Yes Negligible exception,
Control deficiency objective met? not a control
(to Chart 2) deficiency. No further
consideration needed.
Individual boxes should be read in conjunction with the corresponding guiding principles.
10
CHART 2 – Evaluating Process/Transaction-Level Control
Deficiencies
This decision tree is to be used for evaluating control deficiencies, including:
• Design effectiveness deficiencies
• Operating effectiveness deficiencies
• Deficiencies that resulted in a financial statement misstatement detected by management or
the auditor in performing substantive test work.
Step 1: Determine whether a significant deficiency exists.
Box 1. Is the potential magnitude
inconsequential to both annual and
Yes
interim financial statements?
No
Box 7. Would a prudent
Box 2. Are there complementary or official conclude that
Yes Yes No
redundant controls that were tested and the deficiency is at least
a significant deficiency Deficiency
evaluated that achieve the same
control objective? considering both annual
No and interim financial
No statements?
Box 3. Are there compensating
controls that were tested and evaluated Yes
Yes
that reduce the magnitude of a
misstatement of both annual and
interim financial statements to
inconsequential?
No Step 2: Determine whether a material weakness exists.
Box 4. Is the potential magnitude less
than material to both annual and
interimNofinancial statements? Yes
No
Box 5. Are there compensating Box 8. Would a prudent
controls that were tested and evaluated official conclude that
Yes No Significant
that reduce the magnitude of a the deficiency is a
misstatement of both annual and material weakness Deficiency
interim financial statements to less considering both annual
than material? and interim financial
statements?
No
Box 6. Does additional evaluation Yes
result in a judgment that the likelihood Yes
of a material misstatement of both the
annual
No and interim financial statements
is remote?
No
Material
Weakness
Individual boxes should be read in conjunction with the corresponding guiding principles.
11