ABSTRACT

Recent audit failures of unprecedented magnitude and their effects upon the capital markets have resulted in heightened public and regulatory concern towards the auditing profession. Increased scrutiny, combined with a proposed movement away from rules-based accounting standards towards principles-based ones, are creating an auditing environment in which "critical thinking" skills are increasingly important. Consequently, rules-based auditing courses may be insufficient to prepare students for the emerging discipline.

Logic is an important component of "critical thinking;" in fact, the two are considered synonymous in the philosophy literature. Curiously, although logic has been called "the mother discipline of auditing" (Mautz & Sharaf 1961), training in logic is conspicuously absent from accounting curricula, while research in logic is almost non-existent in both the accounting education and audit judgment literatures.

Students in this study were taught formal and informal logic in an auditing course. They studied valid and invalid argument forms within the specific context of auditing services. These students, others without training, and a sample of professional auditors were then tested with a series of real-world auditing vignettes requiring critical reasoning and judgment. Students trained in logic outperformed students without such training. Furthermore, students trained in logic outperformed experienced auditors in their abilities to discern valid versus invalid argument forms. Conversely, experienced auditors outperformed trained students in their abilities to discern believable versus less believable argument premises.

The results of the study have important implications both for university education and for continuing professional education for auditors.

INTRODUCTION AND OVERVIEW

An auditor's commodity in trade is truth. Plato defined truth as a correspondence between what we think and what is real. Thus, the auditor's job is to determine what is and what is not real, then to assure that what is real is properly communicated to the stakeholders. Since determining what is real is not always an easy task, auditors' thinking and reasoning processes are among their most important tools. The ability to accept or reject arguments based on their validity and believability should be a defining competency of a professional auditor.

The accounting education literature is replete with references to "critical thinking," including calls for the improvement of critical thinking skills in accounting students (American Accounting Association 1986; Arthur Andersen, et al. 1989; Accounting Education Change Commission 1990; Deppe et al. 1991, American Institute of Certified Public Accountants 1999; and PricewaterhouseCoopers 2003) and pedagogical techniques purported to improve those skills (e.g., Rodgers 1992; Gainen 1992; Doney and Lephardt 1993; Francis, et al. 1995; Kimmel 1995; Baker et al. 1995; Cunningham 1996; Wolcott and Lynch 1997; Reinstein and Bayou 1997; Bonk and Smith 1998; and Wolcott 1998). Many definitions of "critical thinking" have been proffered (e.g., Doney and Lephardt 1993; Reinstein and Boyou 1997; Baril et al. 1998; Jenkins 1998; and Wolcott et al. 2002). The definitions are generally quite broad, encompassing a wide range of competencies, including both cognitive and non-cognitive skills, attributes, attitudes, and behaviors. Common components of these various definitions include reasoned judgment, ability to identify and solve unstructured problems, professional skepticism, ability to distinguish between facts and claims, lack of bias, etc. Some definitions include characteristics ranging from communication skills to curiosity.

In addition to the lack of a clear, specific and accepted definition of what critical thinking is, this literature on the topic of critical thinking in accounting education has two significant deficiencies. First, there is a lack of empirical evidence that various instructional methods advocated in the literature actually increase the critical thinking skill outcomes of the students, either in general or in accounting-specific situations. Recently, Wolcott, et al. (2002) reviewed this literature and criticized it for a lack of outcomes assessments, concluding that in the absence of such empirical support, the value of efforts to develop students' critical thinking skills must be questioned. Second, this literature is conspicuously devoid of the study of one of the most powerful, commonly-accepted critical thinking tools available; namely, logic; i.e., reaching reasonable conclusions based upon well-founded premises. A thorough review of the accounting education literature yielded no articles that have studied or even specifically mentioned logic as a tool of critical thinking that should be taught to accounting students. This study contributes to the accounting education literature by addressing both of these shortcomings.

In their highly-respected, seminal treatise on auditing, Philosophy of Auditing, Robert K. Mautz and Hussein A. Sharaf (1961) argued that logic is the mother discipline of auditing. However, they did not specifically discuss what logic is, nor did they articulate specifically how it is integral to the discipline of auditing. In the decades since Mautz and Sharaf's monograph was published, auditing educators have occasionally referred to logic in textbooks and articles, but only in passing, and only in a generic sense. A review of current auditing textbooks found only one that specifically teaches formal and/or informal logic and gives examples of valid versus fallacious argument forms (i.e., Ratliff and Reding, 2002).

Similarly, a search of the audit judgment literature revealed a paucity of empirical studies on the role of logic in the auditing process. While the auditing literature contains a plethora of articles on how heuristics and biases, cognitive processing, external factors, and individual differences can affect auditor judgment, there are virtually no articles on whether training in formal logic can mitigate these effects.¹ Those few academic articles which do mention logic (e.g., Power, 1996; Felix & Kinney, 1982) do so only in passing or in a generic sense. Only two practitioner articles were found which discuss logic errors (Jacka 1997; anonymous 1997).

If logic is, in fact, the mother discipline of auditing, it seems reasonable that it might be a valuable component of the common body of knowledge and skill set that should be taught in auditing courses, and also that it could be a topic of interest to accounting researchers. Indeed, the Accounting Education Change Commission's call for teaching critical thinking in accounting courses (1990) may be viewed as a call for teaching logic. While the broad definitions of "critical thinking" offered in the accounting education literature have merit, it is instructive that in the philosophy literature, "critical thinking" is considered synonymous with logic. Critical thinking is defined therein as recognizing and/or constructing sound arguments, applying the principles of formal and informal logic, and avoiding fallacies in reasoning. We believe the ability to evaluate arguments based upon the rules of formal and informal logic is or could be a useful critical thinking tool for auditors.

In summary, based upon our thorough review of the accounting education and auditing literatures, empirical studies have not been done to test the effects of critical thinking instruction that includes rules of logic in accounting and auditing courses. This finding provides the motivation for the research question for this study: "Will training in formal and informal logic help auditors avoid errors in judgment?"

RESEARCH METHOD

This study approaches the research question by exploring the effect of classroom training in formal and informal logic on auditing students' abilities to discern between sound versus fallacious conclusions based on various types of argument forms in audit settings. The performance of students trained in logic is compared with that of students without such training and with experienced, professional auditors.

Two conditions must be met for an argument to be sound or convincing: first, the form of the argument must be valid, meaning the conclusion(s) flow from the premises; and second, the premises must be true. Both validity and truth are required; neither is sufficient alone. This distinction between validity and truth is essential to understanding the constructs examined in this study: No matter how believable the premises may appear to be, an argument is still fallacious if the conclusions do not logically flow from the premises according to the rules of logic; i.e., the argument is fallacious because it is invalid. Conversely, even if the form of an argument is valid, yet the argument will still be fallacious if the premises are not true; i.e., the argument is fallacious because it is based upon false propositions.²

An instrument was designed to assess respondents' abilities to distinguish between sound and fallacious arguments within an auditing context. Specifically, the instrument 1) tested respondents' understanding of the validity of audit-specific arguments and 2) sought insights into respondents' thinking regarding the believability (truth) of the premises -- the former quantitatively and the latter qualitatively.

The authors prepared 29 argument forms, 10 of which were valid or strong inductive forms, and the rest of which were invalid, fallacious, or weak inductions. The authors then met with six audit executives (partner level or equivalent) to gather real-world scenarios structured to the 29 argument forms. Each form was explained to the audit executives, who were then asked to provide actual examples from their auditing careers when they had observed that argument form being used by an auditor or auditee on the job. The executives were not told which of the forms were valid. The audit executives had little difficulty providing examples of all 29 argument forms from their professional experiences, either as participants in the scenarios or by observing others.³ This meeting yielded 174 actual scenarios.

Twenty of the 174 scenarios were selected for the instrument.⁴ Each of the 20 was structured as a short vignette followed by a conclusion reached by an auditor based on the information in the vignette. Respondents were asked to indicate (1) whether the conclusion reached by the auditor in each scenario was "justified" or "not justified" and (2) "how certain" they were of their answers on a four-point scale. A brief explanation of their reasoning for each answer was also requested. According to the rules of logical deduction and induction, seven of the conclusions were justified; the other 13 were not justified.

The twenty scenarios are shown, along with explanations indicating the validity of each, in Appendix A. An example of one of the scenarios is shown below:

A bank's auditors were examining the controls over the processing of loan applications. They knew that if the controls were good, then they would discover few, if any, exceptions in their tests of controls. Indeed, their tests revealed only a few minor exceptions.

Auditors' conclusion: Controls over the processing of loan applications are good.

_____Justified ______Not Justified

How certain are you of your answer? ____I am certain

____I am pretty sure, but not certain

____I think so, but have significant doubts

____I am guessing

Explanation:

The instrument was administered to individuals in the following natural groups:

1. 19 undergraduate students in an auditing course, upon completion of instruction in logic for seven 75-minute class periods.⁵

2. 15 undergraduate students at the end of their first course in auditing, with two class periods devoted to a study of logic.⁶

3. 13 masters' students in accounting at the end of their second course in auditing, with no instruction in logic. (None of these students reported having taken a course in formal logic in any of their studies.)

4. 18 (20 requested, 2 nonresponses) professional auditors with 1-10 years of experience. There was no control for who had or had not taken a course in logic, but because such study is rare for accounting majors, it was assumed that none of the professional auditors had taken such a course.

There is a diverse body of evidence in the general critical thinking literature showing that students who undergo instruction specifically designed to improve thinking skills generally score better on outcome measures than do their counterparts who have not undergone such instruction (e.g., Halpern 1993; Sormunen and Chalupa 1994). Based upon these findings, our first hypothesis is directional:

H1: Student groups with more formal training in logic will outperform student groups with less formal training in logic in identifying justified and unjustified auditor conclusions based on the validity of the argument form.

H1a: Group 1 will outperform Group 2 on correctly discerning validity.

H2a: Group 2 will outperform Group 3 on correctly discerning validity.

Because the literature fails to provide a theory on the effect of professional experience on the development of logic skills and knowledge, our second hypothesis is non-directional (stated in the null form):

H2: There will be no difference between the performance of the student group without formal training in logic (Group 3) and that of auditing professionals (Group 4) in identifying justified and unjustified auditor conclusions based on the validity of the argument form.

A consensus exists in the educational psychology literature that experts employ richer and deeper patterns analyzing information than those employed by novices. For example, Chi et al. (1982) asked advanced Ph.D. and undergraduate students of physics to sort physics problems on the basis of similarities in how they would solve them, without actually solving the problems. They found that novices grouped together problems similar in "surface structure", meaning objects and key words referred to in the problem, whereas experts sorted on the basis of "deep structure", or major physics principles. Expertise, therefore, appears to involve the use of second-order knowledge, abstracted from the problem statement, which is used to direct the solution. Based upon this literature, our third hypothesis predicts experienced auditors will outperform both trained and untrained students in identifying and understanding issues related to the believability of the scenarios:

H3: Auditing professionals (Group 4) will outperform all three student groups (1, 2 and 3) in providing deeper insights into the believability (truth) of the argument premises.

A scoring methodology for the objective measures was agreed upon in advance. Completed instruments were scored independently by an auditing professor trained in logic and a philosophy professor who teaches logic. Respondents' explanations for each answer were carefully examined to determine if any assumption or expansion of the information in the vignette had been made by the respondent that would alter the form of the argument sufficiently to change the appropriate response. When the two sets of scoring results were compared, they were identical.

RESULTS AND ANALYSIS

Hypothesis 1 was confirmed. Group 1 (undergraduates trained in logic) had 88 percent correct responses; Group 2 (undergraduate students with minimal training) had 73 percent correct responses; Group 3 (graduate students without training) had 72 percent correct responses; and Group 4 (professional auditors) had 66 percent correct responses. A one-way ANOVA test was significant beyond the 99 percent confidence level, indicating that the students in Group 1 performed better than the subjects in the other three groups.

There was no statistically significant difference between Groups 2 and 3, suggesting that minimal training in logic does not significantly improve an individual's critical thinking ability. Apparently, substantial training is necessary to make a difference.

Hypothesis 2 (null hypothesis) was not rejected. The raw percentage of correct answers of the professional auditors was lower than that of any of the student groups. Group 4's performance was worse than that of Group 1 at the 99 percent confidence level and also worse than that of Group 2 at the 95 percent confidence level. The difference between Group 4 and Group 3 did not reach traditional levels of statistical significance, but was significant at the 90 percent confidence level.

Another interesting result was the certainty with which both the students and the professional auditors answered their questions. Of the missed questions, Groups 2 and 3 were "certain" their answers were right 50 percent of the time. Group 4 professionals were "certain" of their wrong answers 58 percent of the time. So, not only are auditors untrained in logic frequently subject to errors in critical thinking, but they are likely to reach their wrong conclusions with great confidence, not realizing the likelihood of error.

Hypothesis 3 was confirmed. While fewer of the professional auditors' answers were correct according to the rules of logic relating to the first requirement for the soundness of an argument (validity), they demonstrated superior understanding related to the second requirement for the soundness of an argument (believability/truth). Based on our subjective analysis, Group 4 outperformed all three of the student groups on this dimension. The professional auditors provided more detailed explanations, expressed deeper insights into the scenarios, and demonstrated far more professional skepticism (i.e., willingness to challenge the premises and their underlying assumptions) in the explanations of their answers than did the other three groups.

The following are examples of such explanations from the professional auditors:

"The tests supported the objective of determining whether controls were good. This assumes that the tests were adequate for that purpose."

"Not always does expanding coverage increase labor costs. If dual tests and efficiency are stressed, labor costs could/should stay the same."

"Would need to know if these were weekly meetings, whether it was the manager's responsibility to be there, etc."

"I would reach the same conclusion. However, in the items not sampled, duplicate mailings may exist."

"Other options seem to exist. The company may decide it doesn't want to provide the new product, in which case neither option presented is desirable. Perhaps another product can be eliminated or reduced to free capacity for the new product without increasing facilities."

Although many of these professionals' answers were "wrong" according to the rules of logic, their auditing experience gave them the ability to give the questions more detailed thought, and their frequent expansion of the information presented in the scenarios provided a more professional perspective of the problems. Almost always, however, the amplifications to the scenarios provided in their explanations did not change the validity of the argument according to the rules of logic. Although they thought more carefully about it, their conclusions were still wrong.

In contrast, the student comments exhibited comparatively shallow and simple thinking processes. The following examples of typical student comments indicate an inferior capacity to discern the believability or truth of the vignettes:

"The auditors found what they expected to find from the premise they started with."

"The premise on which the conclusion was based is not very strong, and two people may not draw the same conclusion."

"They can conclude ‘to some customers.' It is hard to determine, when I don't know how the auditors reached their conclusion."

"There isn't any evidence to prove that their time and effort is unnecessary."

"Not a valid conclusion from the circumstances and facts given."

LIMITATIONS AND IMPLICATIONS

Like any study using a quasi-experimentation methodology, this study has some limitations. First, because the subjects were not randomly selected from the populations of interest, unknown differences may exist between the subjects and the population of accounting students in general. Thus, the study has some external validity weaknesses that may limit the generalizability of the results. Second, subjects were not randomly assigned to conditions; individuals were part of natural groups (class sections) and the groups, not the individuals, were assigned to conditions. Thus, the study has some internal validity weaknesses - i.e., selection and selection-history (Cook and Campbell, 1979, 53) - that may limit the interpretation of the findings. For example, the lack of random assignment to groups and the lack of pre-treatment tests means there is no guarantee that the three student groups were equivalent before the treatment. Confounding variables other than training in logic may be responsible for the differences found in the study. Finally, the small sample sizes and the need for corroboration of the findings are important limitations of this exploratory study.

The results of this study have important curricular implications. The study provides reasonably strong evidence that training in formal and informal logic does help auditing students to avoid errors in critical thinking in real-world settings. The magnitude of the difference in performance between groups is fairly large; an investment of approximately 10 hours of instruction yielded an approximately 15 percentage point improvement in performance. Such findings imply that it would be beneficial to include training in formal and informal logic in the accounting curriculum.

This raises the question of incremental cost versus benefit of instruction in logic. Should accounting educators take time to teach logic when there are so many other topics to teach and skills to develop? Isn't 72-73 percent right answers "good enough"? If logic is, indeed, the mother discipline of auditing, we would argue that performance in the "C to C-" range is not sufficient, particularly given the consistent importance placed on critical thinking skills in the literature and the increasing public and regulatory pressures on the profession. Instruction that can increase performance by 1.5 percentage points per hour in one of the most fundamental and important skills required in the discipline is worthy of serious consideration.⁷

Superficial coverage of logic does not appear to have a significant effect. Thus, if they are to include this material in their courses, auditing instructors must be trained in logic so they can teach it in depth. Also, more texts must be revised to include this material. Alternatively, generic logic courses taught by philosophy instructors may be included in the curriculum; however, because such courses do not usually use business, accounting, or auditing situations as examples, further research is needed to determine whether such philosophy courses taught by non-business instructors would have an effect on auditing students similarly beneficial to that found in this study. Additional research is also suggested to replicate and extend the results of this study with larger samples and stronger research designs.

The study also provides interesting research questions for the audit judgment literature. The fact that the audit executives involved in the design stage of the study were able to provide examples for all the various invalid argument forms implies that serious errors in critical thinking and audit judgment, following well-known forms of invalidity, fallaciousness, and other forms of weak reasoning, may not be uncommon among auditing professionals. Furthermore, the fact that the audit professional subjects in the experimental stageof the study were "right" only 2/3 of the time on a dichotomous choice suggests further research into the critical thinking skills of professionals is warranted. For example, empirical studies could examine whether acceptance of fallacious arguments actually leads to incorrect audit judgments. Future research could also explore the question of the importance of logic relative to other critical thinking skills. Finally, it could examine whether 10 hours of instruction would yield significant improvements in the critical thinking skills of professional auditors similar to that of students. If so, continuing professional education programs could be designed to deliver such instruction.

1. For example, articles have examined firm structure (Hermanson 1997); cognitive interference (Anderson & Kaplan 1997); the persuasiveness of evidence (Caster & Pincus 1996), cognitive effort (O'Donnell 1996); personality (Malone & Roberts 1996); recency effects (Cushing & Ahlawat 1996; Trotman & Wright 1996; Messier & Tubbs 1994; Kennedy 1993); auditor experience (Davis 1996); cognitive style (Mills 1996); information ordering (Johnson 1995; Krull & Reckers 1993); Bayesian statistical hypothesis tests (Sennetti 1995; Johnstone 1995); the "curse of knowledge" (Kennedy 1995); anchoring and adjustment (Presutti 1995); linguistic vs. numeric response representation (Stone & Dilla 1994); framing (Emby 1994; Bedard & Graham 1994); and output interference (Anderson & Kaplan 1992). Interestingly, none of these studies examines training in logic as an independent variable.

2. Consider the following argument:
Premise: All cats are dogs.
Premise: I have a cat.
Conclusion: My cat is a dog.
The above argument is valid. A valid argument is one for which the conclusion must be true if its premises are true. If all cats are dogs and I have a cat, then it must be true that my cat is dog. Of course, the conclusion is wrong, but this is because the premises are false; not because the argument is invalid. Validity and truth are different issues.

3. Whether the audit executives made up some of the scenarios - or recalled actual scenarios imperfectly - is not considered a serious limitation to this study. The important point is that the auditors had little difficulty relating each argument form to something that had - or could have - happened on the job; i.e., all the argument forms seemed reasonable and familiar to them. Some forms were more difficult than others, and not every auditor provided a scenario for every argument form, but never did the auditors indicate that a particular argument form was strange or foreign to the auditing environment; each form was viewed as something that could be encountered by a professional auditor.

4. Twenty were chosen for pragmatic reasons: that was the number we could include and have the instrument take less than 20 minutes to complete. The basis for scenario selection was not random. We desired to include a variety of valid and invalid argument forms, and strong and weak inductions. Another consideration was the length of the vignettes, with shorter ones being favored, again to keep instrument administration time at a reasonable length.

5. The instruction was based on Chapters 7 and 8 from Introduction to Auditing: Logic, Principles, and Techniques (Ratliff and Reding 2002). Covered topics included evidence, structure of audit arguments, validity, truth, deduction, induction, eight valid argument forms, sixteen invalid argument forms and informal fallacies, epistemic probability, truth tables, and tests of causality.

6. The instruction covered part of Chapter 7, focusing on validity, truth, and valid vs. invalid argument forms.

7. We believe the cost-benefit of such instruction should be obvious. Consider the following analogy: If medical schools could give instruction for ten hours that would increase the performance of future doctors in correctly diagnosing illnesses from 73 percent to 88 percent, most observers would agree it would be wise to sacrifice 10 hours of instruction in something else to include such instruction in the medical curriculum.

Accounting Education Change Commission. 1990. Objectives of Education for Accountants: Position Statement Number One. Issues in Accounting Education (Fall): 307-312.

American Accounting Association, Committee of the Future Structure, Content, and Scope of Accounting Education ("The Bedford Committee").1986. Future accounting education: Preparing for the expanding profession. Issues in Accounting Education (Spring): pp. 168-195.

American Institute of Certified Public Accountants. 1999. AICPA Core Competency Framework for Entry into the Accounting Profession. New York: AICPA. Available on the Internet at www.aicpa.org.

Arthur Andersen & Co., Arthur Young, Coopers & Lybrand, Deloitte Haskins & Sells, Ernst & Whinney, Peat Marwick Main & Co., Price Waterhouse, and Touche Ross (The Big Eight). 1989. Perspectives on education: Capabilities for success in the accounting profession. New York: Authors.

Anderson, John C. and Steven E. Kaplan. 1997. "The effects of interference and availability from hypotheses generated by a decision aid upon analytical procedures judgments." Behavioral Research in Accounting, 1997 Supplement Conference Papers (9): 1-19.

_______ and _______. 1992. "The effects of output interference on analytical procedures judgments." Auditing: A Journal of Practice & Theory (fall): 1-12.

Anonymous. 1997. "Getting trapped in mental tunnels." The Internal Auditor (Dec): 56-57.

Baker, Richard, Billie M. Cunningham, Paul Kimmel, and Carol Venable. 1995. Critical Thinking. Accounting Pedagogical Resource Series (1995-1) (Karen Pincus - Ed.). Federation of Schools of Accountancy: New York.

Baril, Charles P., Billie M. Cunningham, David R. Fordham, Robert L. Gardner and Susan K. Wolcott. 1998. "Critical thinking in the public accounting profession: Aptitudes and attitudes." Journal of Accounting Education (Summer/Fall): 381-406.

Bedard, Jean C. and Lynford E. Graham Jr. 1994. "Auditors' knowledge organization: Observations from audit practice and their implications." Auditing: A Journal of Practice & Theory (spring): 73-83.

Bonk, Curtis J. and G. Stevenson Smith. 1998. "Alternative strategies for creative and critical thinking in the accounting curriculum." Journal of Accounting Education (spring): 261-293.

Caster, Paul and Karen V. Pincus. 1996. "An empirical test of Bentham's theory of the persuasiveness of evidence." Auditing: A Journal of Practice & Theory (spring): 1-22.

Chi, M. T. H., R. Glaser, and E. Rees. 1982. Expertise in problem solving. In R. J. Sternberg (Ed.) Advances in the Psychology of Human Intelligence. Hillsdale, N.J.: Lawrence Erlbaum.

Cook, Thomas D. and Donald T. Campbell. 1979. Quasi-experimentation: Design and analysis issues for field settings. Houghton Mifflin: Boston.

Cunningham, Billie M. 1996. "How to restructure an accounting course to enhance creative and critical thinking." Accounting Education: A Journal of Theory, Practice and Research (1): 49-66.

Cushing, Barry E. and Ahlawat, Sunita. 1996. "Mitigation of recency bias in audit judgment: The effect of documentation." Auditing: A Journal of Practice & Theory (fall): 110-122.

Davis, Jefferson T. 1996. "Experience and auditors' selection of relevant information for preliminary control risk assessments." Auditing: A Journal of Practice & Theory (spring): 16-37.

Deppe, Larry. A., E. O. Sonderegger, James D. Stice, D. C. Clark, and G. Fred Streuling. 1991. Emerging competencies for the practice of accountancy. Journal of Accounting Education (Fall): 257-290.

Doney, Lloyd D., Noreen E. Lephardt, and James P. Trebby. 1993. "Developing critical thinking skills in accounting students." Journal of Education for Business (68), 297-300.

Emby, Craig. 1994. "Framing and presentation mode effects in professional judgment." Auditing: A Journal of Practice & Theory (supp): 102-115.

Ennis, Robert H. 1993. "Critical thinking assessment." Theory into Practice (32) 179-186.

Felix Jr., William L. and William R. Kinney Jr. 1982. "Research in the auditor's opinion formulation process: State of the art." The Accounting Review (April): 245-271.

Francis, Marlene C., Timothy C. Mulder, and Joan S. Stark. 1995. Intentional learning: A process for learning to learn in the accounting curriculum. Accounting Education Series, vol. 12. Sarasota, FL: Accounting Education Change Commission and American Accounting Association.

Gainen, J. 1992. Developing intellectual skills in accounting education. In T. J. Frecka (Ed.) Critical thinking, interactive learning, and technology: Reaching for excellence in business education. Chicago, IL: Arthur Andersen & Co.: 136-157.

Halpern, D.F. 1993. "Assessing the effectiveness of critical thinking instruction." Journal of General Education 42(4) 238-254.

Hermanson, Heather M. 1997. "The effects of audit structure on auditors'..." Behavioral Research in Accounting: 76-93.

Jacka, J. Mike. 1997. "Door number three." The Internal Auditor (Dec): 54-57.

Jenkins, Elizabeth K. 1998. "The significant role of critical thinking in predicting auditing students' performance. Journal of Education for Business (May/June): 274-279.

Johnson, Eric N. 1995. "Effects of information order, group assistance, and experience from auditors' sequential belief..." Journal of Economic Psychology (March): 137-160.

Johnstone, David J. 1995. "Statistically incoherent hypothesis tests in auditing." Auditing: A Journal of Practice & Theory (fall): 156-175.

Kennedy, Jane. 1995. "Debiasing the curse of knowledge in audit judgment." The Accounting Review (April): 249-273.

_______. 1993. "Debiasing audit judgment with accountability. A framework with experimental results." Journal of Accounting Research (fall): 231-145.

Kimmel, Paul. 1995. "A framework for incorporating critical thinking into accounting education." Journal of Accounting Education (summer): 299-318.

Krull Jr., George and Philip M.J. Reckers. 1993. "The effect of experience, fraudulent signals and information presentation order on auditors'..." Auditing: A Journal of Practice & Theory (fall): 143-153.

Malone, Charles F. and Robin W. Roberts. 1996. "Factors associated with the incidence of reduced audit quality behaviors." Auditing: A Journal of Practice & Theory (fall): 49-64.

Mautz, Robert K. and Hussein A. Sharaf. 1961. Philosophy of Auditing. Sarasota, FL: American Accounting Association.

Messier Jr., William F. and Richard M. Tubbs. 1994. "Recency effects in belief revision: The impact of audit experience and the review process." Auditing: A Journal of Practice & Theory (spring): 57-72.

Mills, Tina Y. 1996. "The effect of cognitive style on external auditors..." Behavioral Research in Accounting: 49-73.

O'Donnell, Ed. 1996. "Measuring cognitive effort during analytical review: A process-training framework." Auditing: A Journal of Practice & Theory (spring): 100-110.

Power, Michael. 1996. "Making things auditable." Accounting, Organizations and Society (Feb/Apr): 289-315.

Presutti Jr., Anthony H. 1995. "Anchor and adjustment heuristic effect on audit judgment." Managerial Auditing Journal (9): 13-22.

PricewaterhouseCoopers. 2003. Educating for the public trust: The PricewaterhouseCoopers Position on Accounting Education. New York: Author.

Ratliff, Richard L. and Kurt F. Reding. 2002. Introduction to Auditing: Logic, Principles, and Techniques. Altamonte Springs, FL: The Institute of Internal Auditors (ISBN 0-89413-472-8).

Reinstein, Alan and Mohamed E. Bayou. 1997. "Critical thinking in accounting education: Processes, skills and applications." Managerial Auditing Journal (July): 336-342.

Rodgers, R. F. 1992. Teaching to facilitate critical thinking: A cognitive developmental perspective. In T. J. Frecka (Ed.) Critical thinking, interactive learning, and technology: Reaching for excellence in business education (91-113). Chicago, IL: Arthur Andersen & Co.

Sennetti, John T. 1995. "On the incoherent use of evidence: Why subjective Bayesian evidence is not held probative." Auditing: A Journal of Practice & Theory (fall): 193-199.

Sormunen, C. and M. Chalupa. 1994. "Critical thinking skills research: Developing evaluation techniques," Journal of Education for Business 69(3): 172-177.

Stone, Dan N. and William N. Dilla. 1994. "When numbers are better than words: The joint efforts of response representation and experience." Auditing: A Journal of Practice & Theory (supp): 1-19.

Trotman, Ken T. and Arnold Wright. 1996. "Recency effects: Task complexity, decision mode, and task..." Behavioral Research in Accounting: 175-193.

Wolcott, Susan K. and Cindy L. Lynch. 1997. "Critical thinking in the accounting classroom: A reflective judgment developmental process perspective." Accounting Education: A Journal of Theory, Practice and Research (1): 59-78.

Wolcott, Susan K., Charles P. Baril, Billie M. Cunningham, David R. Fordham, and Kent St. Pierre. 2002. "Critical thought on critical thinking research." Journal of Accounting Education 20(2): 85-103.

Wolcott, Susan K. 1998. "Critical thinking development in the accounting curriculum: Focusing on ambiguity in introductory accounting courses." In D. F. Fetyko (Ed.) Changes in accounting education: Implementation in specific accounting courses and subject areas (1-16). St. Louis: Federation of Schools of Accountancy.

Following are 20 scenarios, each giving the conclusion reached by the auditor or auditors, or another person, involved. Each scenario is taken from a real audit situation, with some situational facts changed or omitted to preserve the propriety of the information.

For each scenario, please indicate with a check mark in the appropriate space whether you think the conclusion is justified or not justified by the information in the scenario. Also, please place a check mark in the space corresponding to your level of certainty on the answer given. After each response, please explain briefly your answer. Please answer every question. The entire survey should take no more than 30 minutes to complete.

1. A bank's auditors were examining the controls over the processing of loan applications. They knew that if the controls were good, then they would discover few, if any, exceptions in their tests of controls. Indeed, their tests revealed only a few minor exceptions.

Auditors' conclusion: Controls over the processing of loan applications are good.

_____Justified ______Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is not justified. The argument is (simplified):

If the controls are good, then there are few exceptions in the tests of controls.
There are few exceptions in the tests of controls.
Therefore, the controls are good.

But the controls need not be good, even though few exceptions were found in the tests. Perhaps the tests were not run correctly or carefully, for example. These alternative explanations would account for finding few exceptions. This argument is of the fallacious form Affirming the Consequent:

If p then q
q Therefore p

2. A new staff auditor was assigned to examine some documents kept in a filing room. The controller explained to the new auditor that "the filing clerk is badly overweight, unkempt, unfriendly, and generally is unpleasant to be around." The controller concluded with the following statement:

Controller's conclusion: "You will find the files to be in disorder."

_____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The controller's conclusion is not justified. One can readily detect the irrelevant personal attack. This is an example of the fallacy Ad Hominem ("to the man"). The filing clerk's personal appearance and degree of friendliness are not of concern here. We are concerned, rather, with whether or not the clerk's files are orderly. The former does not provide relevant evidence for evaluating the latter.

3. An auditor observed one of the client's accounting clerks preparing the month's bank reconciliation.

Auditor's conclusion: Monthly reconciliations are performed.

____Justified ____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is not justified. Observing the preparation of only one month's bank reconciliation does not justify the claim that, in general, monthly reconciliations are performed. The sample is too small. This is an example of the fallacy Hasty Generalization.

4. An audit manager missed an audit team meeting, because he did not know it was scheduled. When he saw the audit team leader afterward, he asked:

Manager's question: "Why didn't you notify me about the team meeting this morning?"

____Justified ____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The manager's question is not justified. Within this question is buried the assumption that the audit team leader did not take the proper steps to notify the manager. This should not simply be assumed, however. Perhaps the team leader did take the proper steps, but for some reason the notification did not reach the manager. Perhaps a secretary did not pass the notification along to the manager, for example. This is an example of the fallacy Complex Question.

5. The director of internal auditing met with the lead auditor of the assignment. The two were discussing whether to expand the coverage of the audit. The director explained that if coverage were expanded, then more staff would have to be assigned to the audit; and, he said, if more staff were assigned, then labor costs would increase for the audit.

Lead auditor's conclusion: If coverage of the audit is expanded, then labor costs will increase for the audit.

____Justified ____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The lead auditor's conclusion is justified. This is a clear example of the valid argument form Hypothetical Syllogism:

If p then q
If q then r
Therefore if p then r

The conclusion properly links the antecedent of the first premise, p, with the consequent of the second premise, r, through the common sentence, q.

6. A team of state auditors was examining the state's department of transportation, which was installing a new state-of-the-art computerized information system. The system development team and department employees who would be operating the system all were confident the new system would work. The employees had been working with the development team from the beginning of the project. None of these experts doubted the immediate success of the new system.

Controller's conclusion: There is no need to run the old information system parallel with the new system after installation.

____Justified ____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The controller's conclusion is not justified. This is an explanation of the fallacy known as Subjectivism or Wishful Thinking. The controller is arguing from the premise that the experts believed the new system would work, to the conclusion that it really would work. Considering the complexity of a new computerized information system, however, the simple belief that it will work provides poor support for the claim that it really does. The new system should be tested.

7. The internal auditors for a public utility company performed an audit of the company's billing system in 1995. They recommended that the addressing process be fixed to avoid an existing problem with duplicate mailings to some of the company's customers. The internal auditors returned in 1996 to perform an audit of the same system. The results of the audit did not indicate a problem with duplicate mailings to customers. The lead auditor was reviewing the work of the assigned auditors.

Lead auditor's conclusion: The problem with duplicate mailings to some customers has been fixed.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The lead auditor's conclusion is not justified. This is an example of the fallacy Appeal to Ignorance. We know only that the later audit did not indicate a problem with duplicate mailings. This provides poor support for the conclusion that the problem has been fixed. Possibly the later audit was done incorrectly. Possibly the later audit did not even consider the issue of duplicate mailings!

8. Auditors beginning an audit of a chain of restaurants were doing a preliminary review of engagement with the client. One of the issues of concern was legal matters that could affect contingent legal liabilities. The auditors sent a written inquiry to the company controller as a part of this preliminary audit work. The controller responded to the written inquiry, including a response that there were no legal matters of concern.

Auditors' conclusion: Risk related to contingent legal liabilities is low.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is not justified. This is the fallacy of Appeal to Inexpert Authority. The company controller is presumably not an expert in legal matters. The auditor should seek the opinion of the company's attorney.

9. In a closing conference with an auditee, an internal auditor explained the following:

"Either you can manufacture this product, or you can outsource the manufacturing process (i.e., the plant could hire another company to manufacture it). If you manufacture the product, you will have to increase your facilities. If you outsource the manufacturing process, then you must find a reputable supplier to rely on the work of others."

Auditor's conclusion presented to plant management: "Therefore, either you must increase your facilities, or you must find a reputable supplier and rely on the work of others."

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is justified. This is a clear example of the valid argument form Constructive Dilemma: If p then q
If r then s
p or r
Therefore q or s

Given the truth of if p then q, and if r then s, and given the truth of the disjunction of the antecedents, p or r, it must be the case that q or s.

10. An auditor discovered no exceptions in any of 600 randomly selected cash disbursements tested, of a possible 3,000.

Auditor's conclusion: Cash disbursements are being recorded properly.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: This item was purposely structured in such a way that auditor's conclusion may or may not be justified, depending upon the assumptions made. If less conservative assumptions are made, six hundred randomly selected cash disbursements is a good sample for testing whether all 3,000 cash disbursements are recorded properly, and the auditor's conclusion would be justified. However, using very conservative assumptions, 600 would provide only slightly more than a 90 percent confidence level. If a 95 percent confidence level were desired, the conclusion would not be justified. This item was graded as follows: if no explanation was given or if the explanation failed to address issues of statistical confidence, the "correct" answer was "justified." If the explanation addressed issues of statistical confidence in such a way that the definition of a valid induction was raised to a higher level, answers of "not justified" were accepted as correct.

11. Auditors of a branch of a retail chain were examining cash receipts by clerks working at the branch. The auditors reasoned that if one or more of the clerks had misappropriated cash receipts, then cash deposits would be short. The auditors' test showed that the deposits were not short.

Auditors' conclusion: The clerks did not misappropriate cash receipts.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditors' conclusion is justified. This argument is of the valid form Denying the Consequent or Modus Tollens:

If p then q
not q
Therefore not p

Given that the deposits were not short, it must be the case that the clerks did not misappropriate cash receipts. For if they had misappropriated cash receipts, the cash deposits would have been short, according to the first premise. And we know that the deposits were not short. To avoid this contradiction, we must admit that the clerks did not misappropriate cash receipts.

12. The management of one financially troubled client of an accounting firm was worrying about the company's declining profits. In a meeting with the audit team leader, the company CEO posed the following dilemma:

CEO's conclusion: We can either cut costs or go broke.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The CEO's conclusion is not justified. This is an example of the fallacy known as False Alternatives. The conclusion presents only two alternatives, leaving out alternatives that should be considered. Perhaps the company could increase advertising, for example.

13. The staff auditor assigned to a warehouse engagement was performing some end-of-period tests on cash receipts for sales. The auditor knew that cash deposited on January 3 was either revenue for the prior year or for the current year. Testing the deposit, she verified that the deposit was not revenue for the current year.

Auditor's conclusion: The deposited cash was revenue for the prior year.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is justified. This is an example of the valid argument form Disjunctive Syllogism. There are two versions:

p or q p or q
not p not q
Therefore q Therefore p

Given the truth of a disjunction, p or q, and the denial of either sentence in the disjunction, the other sentence in the disjunction must be true.

14. A performance auditor for an accounting firm was performing an audit of a manufacturing plant, evaluating the plant's capacity to produce a particular product. The auditor's analysis demonstrated that the client's plant had the capacity to manufacture the product.

Auditor's conclusion: The client could produce the product itself or outsource the manufacturing process from another company.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is justified. This is an example of the valid argument form Addition:

p
Therefore p or q

Given the truth of any sentence p, a disjunction of that sentence with any other sentence, p or q, must be true. In the conclusion, the truth of p forces the disjunction p or q to be true.

15. A frustrated audit manager entered his supervising partner's office and said, "That audit team on the [name of company] audit are inexperienced and spend their time nitpicking."

Management's conclusion: "The audit is unnecessarily inefficient and unfairly critical."

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: Another irrelevant personal attack. The management's conclusion is not justified. This is the fallacy of Ad Hominem ("to the man"). Once the audit is in place with results that can be evaluated, it is the audit that is of concern, not the personal characteristics of the audit team. Perhaps the audit is inefficient and unfairly critical, but relevant evidence of this is not provided by pointing out the lack of experience of the team and their "nitpicky" nature.

16. An auditor working on an audit of a government office was assigned to examine cash expenditures. He determined that if there were segregation of duties in the cash disbursements process, then it would be difficult for an employee to commit a fraud against the system. He discovered that the process did not employ segregation of duties.

Auditor's conclusion: It would not be difficult for an employee to commit a fraud against the system.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is not justified. This is an example of the invalid argument form Denying the Antecedent:

If p then q
not p
Therefore not q

17. Auditors examined two journal entries made by an accounting clerk for debt. Both entries were faulty.

Auditors' conclusion: All of the clerk's debt entries are faulty.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditors' conclusion is not justified. Only two faulty debt entries does not provide good support for the conclusion that all the clerk's debt entries are faulty. This sample is too small, and perhaps represents simply a bad day for a generally good clerk. This conclusion is a Hasty Generalization.

18. An auditor was examining the medical charges claimed by customers of an insurance company. The auditor determined from his tests that if the medical charges were properly coded by the company, then reimbursements would be correct. He further determined that medical charges were coded correctly.

Auditor's conclusion: The reimbursements were correct.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is justified. This is a clear example of the valid argument form Affirming the Antecedent or Modus Ponens:

If p then q
p
Therefore q

Given that the medical charges were coded correctly, then by the first premise of the auditor's argument it must be true that the reimbursements were correct.

19. A team of internal auditors was examining the performance of manufacturing operations for a Canadian plant of a large U.S. company. The plant introduced a new product in April. Customer sales returns at the plant increased dramatically in May and June.

Auditors' conclusion: The new product probably will be unsuccessful if nothing is done.

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The auditor's conclusion is not justified. This is an example of the fallacy known as False Cause. We know only that customer sales returns at the plant increased dramatically after the introduction of the new product. It does not follow that the new product caused the dramatic increase in sales returns.

20. An audit manager was interviewing an MIS manager about the MIS control system. A portion of the conversation went as follows:

MIS manager: "MIS controls have been implemented."

Audit manager: "Our examination shows that application controls have not been implemented."

MIS manager: "I agreed to implement the programming controls, and those controls have been implemented."

Audit manager's conclusion: "MIS controls have been implemented."

____Justified _____Not Justified

How certain are you of your answer? ____I am certain
____I am pretty sure, but not certain
____I think so, but have significant doubts
____I am guessing

Explanation: The audit manager's conclusion is not justified. In response to the comment that application controls were not implemented, the MIS manager responds that programming controls were implemented. This is an attempt by the MIS manager to divert attention away from the issue at hand, which is whether or not full MIS controls, including application controls, have been implemented. This attempt at diversion is made less obvious by the fact that programming controls are by definition part of MIS controls. This is an example of the fallacy known as Definitional Dodge. It is also a good example of the broader fallacy called Diversion.

To read other articles by Irv Nelson, link here.