Much literature is available on programs to teach critical thinking, and a substantial amount of evidence indicates critical thinking can be taught and learned, especially when instruction is specifically designed to encourage transfer of skills. Nevertheless, the types of studies required to confirm with certitude the efficacy of teaching critical thinking present practical and methodological problems.
Most definitions of critical thinking refer to the mental processes of reasoning logically, making judgments, questioning, and reflecting on the process itself. I define the term in the following manner: Critical thinking is the use of those cognitive skills or strategies that increase the probability of a desirable outcome. Critical thinking includes evaluating the quality and outcome of the thinking process.
A Skills Approach
Critical thinking instruction that is skill based has specific educational objectives-and thus is easier to assess and communicate to students and other stakeholders- and provides a framework to focus classroom lessons. Some examples of thinking skills, applicable in a wide range of situations, are understanding how cause is determined, recognizing and criticizing assumptions, analyzing means-goals relationships, supplying reasoned support for conclusions, assessing probability, incorporating isolated data into a wider framework, and using analogies to solve problems.
Thinking skills can be taught and transferred to other topics. Transfer is the spontaneous use of a skill in a context different from the one in which it was learned and is the goal of critical thinking instruction. The failure to transfer a skill maybe attributed to inadequate learning of the skill or teaching that does not encourage transfer. When skills are taught for transfer-with multiple examples across different domains of knowledge, uncued but with corrective feedback-they do transfer. Such teaching should include direct instruction with review, teacher modeling, guided and spaced practice, and independent application.
Assessment as an Operational Definition
The assessment of an intervention is almost as important as the intervention itself. Assessment is tied to issues of definition, research design, and essential debates over whether it is possible to improve thinking. When the measurement is bad, it is easy to see why we have not gotten strong results with critical thinking instruction, but the measurement issues in critical thinking are not insurmountable.
A Better Measure
The need for providing information about the status of critical thinking skills is relatively uncontroversial. There is currently little information to inform decision makers concerned with improving thinking skills. The controversies arise over questions of whether the information can be provided in a way that is meaningful, valid, fair, and cost effective. If the assessment is not well done, the results will be costly. A good measure of critical thinking would be based on clearly defined skills assessed in realistic scenarios that could apply to a wide range of ethnic and socioeconomic groups. The skills selected must be ones used in most cultures.
The Sequential Question
The critical thinking uses realistic examples with an open-ended response format, allowing participants to demonstrate spontaneous use of skills. Participants are then probed for alternatives in forced-choice questions, demonstrating their understanding of concepts and showing if they are able to use skills when prompted. A good critical thinking question with several sequential parts allows for different types of information about participants with a minimal number of questions. Open ended parts test “free recall” because they place few restraints on responses. Multiple-choice parts show if respondents are able to recognize skills presented in a list, a measure of “recognition memory.” These two types of recall use different cognitive processes. Lower scores are expected on free recall tests because they require a search through memory plus a verification of answers; recognition requires only the verification stage.
Tests presented on computers provide reaction time data, which help provide information about the microcomponents of the underlying cognitive processes. Reaction times permit a much more fine-grained analysis of mental events than other commonly used dependent measures.
Cognitive psychologists can now provide sufficient knowledge of how people think, learn, and remember. People retain information best when they generate information from memory, space practice over increasing time intervals, remain active, receive informational and useful feedback, and use visuospatial and verbal formats.
Thorny Conceptual Issues
The literature on teaching thinking skills is huge but difficult to summarize statistically because of the variety of instructional strategies-team teaching, learning hierarchies, tutoring, questioning, and concept mapping, to name a few-that have been investigated. Random assignment field trials may be proposed as a way to confirm the efficacy of teaching critical thinking, but this supposition is based on imperfect analogy between education and medicine: We do not improve thinking the same way we prevent polio. Furthermore, the many criticisms of null hypothesis testing cannot be “fixed” by randomly assigning participants to conditions. Alternatively, meta-analyses might allow for information across studies to be considered along with a single estimate of their effect size, but such meta-analyses raise the issue of how multiple studies with large effect sizes with a matched control group should be weighed against a single experiment with random assignment of subjects and a smaller effect size. One synthesis of studies of thinking skills programs computed an overall effect size of d = 1.17 from 45 separate effect sizes. With an effect size over one standard deviation across studies with diverse subjects and settings, as was the case in this synthesis, do we need large random assignment field trials before we can decide that these interventions work to improve thinking skills? Furthermore, would informed parents allow their children to be in the control or non-treatment group in a randomized trial? The “correct” response is that without random assignment, it is not possible to know if the intervention actually worked. However, a large effect size summarized over a large number of diverse studies from many different participants and contexts also provides good evidence, even if it is not strictly causal.
Avoiding Design Flaws
Even though they are inherently flawed, we absolutely need large-scale random assignment studies, but we need to be mindful of their limitations and not blindly accept conclusions as “the answer” to questions. We can also use meta-analyses that indicate effect sizes and other types of converging evidence. The complexities of real children in real learning environments do not easily lend themselves to the manipulation of single variables under controlled conditions, but these sorts of studies need to be funded and encouraged or they will not happen because of the necessary expenses and need for replication, fidelity, and collaboration.
Strong Causal Evidence
One large-scale, double-blind, random assignment experiment of a thinking skills intervention showed that targeted thinking skills were transferred and used appropriately with novel topics. Students who received the thinking skills instruction showed greater gains than control group students on tests of general aptitude, problem solving, decision making, reasoning, creative thinking, and language.
Large-scale studies of the type described above are expensive and need governmental or foundational support, but such studies are needed so that results can be replicated across sites and so researchers can establish necessary controls to determine the effect sizes. Conclusions from studies with poor controls suggest that low-achieving students make the greatest gains, perhaps because they have the greater possible latitude for additional cognitive gains, but experiments are needed to verify this.
Students can think better as a result of instruction, but we lack the strongest causal data with longitudinal follow-up. More randomized field trials are needed. These studies are expensive and difficult to coordinate but are worth the investment. Educated adults need to be able to judge the credibility of information, recognize and defend against propaganda, reason effectively, use evidence in decision making, and identify problems and find solutions if they are to benefit from the wealth of available information. Doing all this may be the best return on investment we make as a nation.