Month: April 2016

I. Logicism and normativism and their disconnects

Norms play a special role in mental processing- reasoning, judgment, and decision making- and direct us in how we ought to behavior. There are two types of normativism: empirical and prescriptive (234).

–Empirical normativism: Thinking reflects S
–Prescriptive normativism: Rational thinking should be measured against
S as a normative system, and ought to conform to it.
*S represents any type of normative system including logic, Bayesian
probability, or decision theory.
*”normativism” can be substituted with with other formal normative
systems such as logicism or Bayesianism.

These two tenets make normativism a system of varying degrees (diagram on 235). Researchers vary in where they stand in across both the prescriptive and empirical spectrum, however, tend to adhere to high prescriptive norms. Interestingly, there are no individuals who adhere to high empirical norms and low prescriptive norms simultaneously. Many factors, including the role of a priori knowledge in normativism or whether or not a normative system is.

E&E’s main problem is with prescriptive normativism, arguing that it is a problematic system and unnecessary in scientific studies regarding human thinking (235). In their article, they touch on what they mean by normativism, the problem of arbitration, the is-ought inference and its role in two research programs in human thinking, how normativism creates negative bias in the study of human thinking, and why normativism is unnecessary. In conclusion, they suggest that a descriptive approach should be taken when addressing mental processes.

II. Normativism, rationality, and the three senses of ought

There are a number of different types of rationality including instrumental, bounded, ecological, and evolutionary rationality. Normative rationality is yet another form of rationality and is unique because it implies what we ought to do. There are three main meanings for ought (236):

1. Directive Deontic:  Selection-for and functional analysis. You must turn left at the light.
2. Evaluative Deontic: Normative. You should not steal.
3. Epistemic: Expressing belief of probability – He should be able to catch his flight.

The normative ought is evaluative. The distinction between different forms of ought have aided in the controversy over instrumental rationality (acting in a way to achieve one’s goals). Some philosophers, like Oaksford and Chater (O&C), don’t see the distinction between the evaluative and directive ought. Instead, they view instrumental rationality as something that should be justified through norms, thus blurring the lines between instrumental and normative rationality. However, E&E disagree with this and believe that there should be a clear distinction between these two types of rationality. They also take the stance that normativism isn’t necessary or useful when discussing function, adaptation, and ecological and instrumental rationality: “if behavior is typically adapted and we typically achieve personal goals, we should be rational without the use of normativism” (236).

III. Normative systems and the problem of arbitration

Each paradigm requires a particular normative system, and only the one right and appropriate norm can be applied.

Despite the need for one right system, there is no universal “clear-cut norm,” and this weakens normativism (237). Normativism is weakened further if there happens to be multiple norms that fit into a paradigm. This phenomenon is called the normative system problem or inappropriate norm argument.

There are three norm paradigms: single, alternative, and multiple (237).
–Single: Only one norm can be applied. Examples include signal detection
task and conditional elimination inference. No conflict.
–Alternative: One standard norm and at least one alternative can be applied. Examples include conditional induction inference and the Wason selection Task. Conflict.
–Multiple: There are several, equally standard norms that can be applied. Examples include metadeduction. Conflict.

Single norm paradigms are common in situations such as memory task, however, they are rare in reasoning and decision making scenarios. This paired with the frequency of alternative and multiple norm conflict pose a major challenge for normativism.

IV. The computational, the competent, and the normative

E&E’s main argument is not a complete rejection of the formal systems in exchange for processing ones. Instead, they reject the use of these formal systems as normative ones and reject the deontic, evaluative ought. E&E draw a distinction between normative theory and competence theory in order to highlight the distinction between ought and is. The competence theory includes Chompsky’s and Marr’s parallel definitions of competence and computational levels of analysis.  For Chompsky, competence is the “structural description of abstract knowledge that is value free,” or rather looks to answer the question ‘what is…? (239). Likewise, Marr’s definition of computational levels of analysis is something that describes what is being computed and why, or rather answer the question ‘how is…?’ Both of these stances help make up the computational/competence theory (descriptive theories), while the question of what ought to be makes up the normative theory. E&E believe it is critical to discern between these two theories and between is and ought. Without a clear distinction between descriptive theories and normative theories, people fall into a controversial type of inferences: inferring ought from is.

V. Inferring ought from is

Normativism functions selectively and and there is one “appropriate” norm to use for a given scenario. However, this becomes increasingly difficult in alternative and multiple norm scenarios where there can be one or many alternative norms. Choosing the correct norm is difficult but also extremely crucial for normativism to function properly.

Challenging normative theories are competence theories. Competence theories are descriptive and supported by descriptive evidence. E&E and other researchers believe that descriptive evidence cannot be used in normative instances. However, when people derive a normative conclusion from descriptive premises, they are said to follow the is-ought inference. Often times, the is-ought inference includes an implicit normative premise. The implicit normative premise is an internal belief that “we should  act in line with our natural instincts” (240). The issue with this is that if this premise is not made explicit, the argument becomes fallacious. Similar to the is-ought inference, the naturalistic fallacy draws evaluative conclusions from natural events. Specifically, the naturalistic fallacy occurs when ethical norms are inferred from natural phenomena such as evolution.

V.1 Oaksford and Chater’s (O&C)  Bayesian rational analysis: O&C proposed that logicism, both empirical and prescriptive, should be rejected and replaced by Bayesianism. Bayesianism suggests that human thinking is based on Bayesian probability and normatively justified by it. However, E&E oppose O&C on the level  that they do not separate is from ought. O&C support the circle of normativity, where everyday rationality (successful behavior, instrumental) is justified by formal rationality (logic, Bayesian probability, etc). Furthermore, everyday rationality provides empirical evidence to chose the formal normative system. However, the empirical evidence does not clearly state which normative system to use. E&E have difficulty with O&C’s complex model as it uses a number of different oughts and uses the is-ought inference.

V.2 The individual differences programme of Stanovich and West: S&W’s earlier work also follows the is-ought inference by connecting normative and computational-level analysis. S&W suggests that correct reasoning is due to higher quality reasoning and cognitive capacity, as higher ability patients gave better answers to particular tasks in past studies. Therefore, the system the higher ability people endorse is the correct system and should become the norm. However, this falls into the trap of the is-ought inference. The is  in this case is that higher ability people chose more correct answers and the ought is that the norm they utilized is the correct one.

V.3 Evaluative ought versus directive ought: O&C rational analysis and S&W’s earlier work both incorporate the is-ought inference, however, they draw different normative conclusions. O&C focuses on adaptationist learning and suggests that ‘gene-directed behavior’ is rational. On the other hand, S&W focuses on the self-described Meliorist approach and suggests that people are not innately rational but can learn to be rational through education and training (242). This distinction can be seen in O&C’s and S&W’s interpretation of the Wason selection task. S&W suggest that logic is the best normative system  for the task whereas O&C suggest that information theory is the best norm to use. S&W chose logic because higher ability individuals solve the problem through logic, and O&C chose information theory because the majority of individuals use it during the task.

VI. Normativist research biases

E&E argue that normativism has triggered three types of research bias in psychologists’ approach to studying human reasoning, thinking, and JDM (Table 3). First, they use a logic and deduction paradigm and Bayesian rationality to introduce the prior rule bias. E&E show that after untrained individuals are instructed to accept a new normative system of logic, their thinking will reflect this “built-in” normative system (empirical normativism). This built-in system ultimately constrains reasoning and may permit participants to rationally get answers wrong (243). Secondly, they introduce interpretation bias, which suggest that normativism has negatively influenced the way results are recorded and interpreted. Evans suggests that to avoid bias we should record exactly what people do without interpreting their logical accuracy or concern of what they ought to do. The ought-is fallacy is an interpretation bias involving the normative account of dual-process theories of reasoning. The ought-is fallacy assumes that System 2 is responsible for correct normative responding, while System 1 (ie. heuristics) is associated with cognitive bias. This a situation where System 2 (i.e. analytical) is involved in inferring is from ought. Some authors suggest that System 2 is “necessary” for normative rationality while E&E suggest this “rule-based reasoning” may actually result in normative error (245). Thirdly, the clear norms bias proposes that psychologists are biased to select research questions involving single-norm paradigm even though they are rare and largely inapplicable to questions regarding JDM. In response to these problems, E&E suggest that adopting a descriptivist approach in place of a normativist approach would help eliminate these biases.

VII. Can we manage without a normative theory?

The previous sections of this article, E&E have identified the problems of normativism, however, they acknowledge the fact that normative based formal systems have motivated several valuable and productive psychological research paradigms (i.e. Wason selection, 2-4-6 task, etc.). These formal theories have a range of important relations to psychological theories shown in Fig. 2. Given the “heuristic value” of formal theories, E&E argue that descriptivism is a viable alternative to normativism because it can maintain these important relations to psychological theories, without problematic inferences and research biases (246).This may be achieved by a dual-process framework dubbed hypothetical thinking theory (HTT). HTT extract the aspects of subjectivity, belief, and uncertainty from Bayesian theory and proposes that System 2 is capable of hypothetical thinking using epistemic models.

Conclusions

E&E conclude that a normativist approach to the psychology of reasoning and JDM is both probabilistic and unnecessary. Instead we should utilize a descriptivist approach as an alternative to normativism in order to avoid research biases and circular is-ought inferences. However, it should be noted that norms can play a role in applied sciences and research regarding planning, policy, and development.

Discussion Questions:

1. In section II E&E discuss the question regarding function and normativism (heart example). Do you agree with E&E that function falls under a different type of ought than normativism, or do you think that function should be classified with normativism?
2. If E&E warn against inferring ought from is, where does the concept of ought originate from?
3. E&E highlight the arbitration problem as something that weakens normativism, especially because of the limited application of the single-norm paradigm. Can you think of an instance (not mentioned in the article) in which single-norm paradigm can be useful?
4. Although E&E do not suggest the complete elimination of normativism, do you think it is possible to approach reasoning and JDM without a normative theory?

• Allott & Uchida
  • O&C claim that heuristics that involve information gain should be used. Allott & Uchida state that classical logic and O&C’s probabilistic account of conditionals and of inference must be supplemented by accounts of processing.
• Brighton and Olsson
  • O&C discuss rational analysis as a process model used to develop optimal behavior. However, Brighton and Olsson believe that functional analysis can occur without a need for optimality.
• Danks & Eberhardt
  • Danks & Eberhardt agree with O&C that the teleological explanations of human behavior are desirable, but they need a stronger foundation. They attest that Bayesian inference is neither a normative principle nor a subject of optimality as a result of people approximating explanations.    
• Neys
  • Neys shares that O&C’s modeling has an exclusive focus on output data which could lead to biased conclusions. He indicates that people are constantly trying to meet the norm.
• Evans
  • O&C state that individuals resemble Bayesian reasoners more closely than standard logic. Evans agrees that the Bayesian model is better for real world reasoning than one based on truth-functional logic. However, Evans doesn’t know why O&C need to fit a normatively rational model to human reasoning.
• Griffiths
  • Griffiths further examines the strengths and weaknesses of Bayesian models of cognition. Strengths include the systematicity of rational explanations, transparent assumptions and combining symbolic representation with statistics. Some of the challenges include providing psychological mechanisms, explaining origins of knowledge and describing how people make new discoveries.
• Hahn
  • Hahn believes that an increase in explanatory power can be achieved by restricting a psychological theory. Although cognitive neuroscience experiments can lead to results, they are not as significant because of the successful opposite trend of O&C.  
• Halford
  • O&C believe that confidence is a function of informativeness. Halford counters that confidence is inversely related to complexity and that Bayesian rationality should be replaced by principles regarding cognitive complexity.  
• Khalil
  • Khalil examines the question of rationality and whether humans use classical deductive logic or probabilistic reasoning. He attests that organisms do process information and respond to the environment in ways that qualify them as rational.
• Liu
  • Liu proposes that conditional probability hypothesis exists only when reasoners explicitly evaluate probability of conditionals, but that it may not exist when making (MP) inferences.  
• McKenzie
  • O&C suggest that deductive reasoning is parsimonious at a local and global level. They focus on environmental structure at the computational and algorithmic levels.
• Nelson
  • Nelson believes that naive heuristic strategies can perform better than “optimal models.” Thus the normative role of the theoretical model and the adaptiveness of human behavior should be reexamined.
• Oberauer
  • O&C state that people use probabilistic information to reason with unknown information. Oberauer believes that the probabilistic view on human reasoning has high a priori rationality and that that data by O&C is ambiguous.
• O’Brien
  • O&C have rejected logic and supported probability theory. O’Brien explains that the mental-logic theory is based on logic that developed through bioevolutionary history to gain an advantage in making simple inferences.
• Over and Hadjichristidis
  • O and H have an issue with O&C’s assumption that minor premises in conditional inferences are always certain, and believe that Jeffrey’s rule is not limited enough to account for actual probability judgements.
• Pfeifer and Kleiter
  • They address O&C’s probabilistic approach from a probability logic standpoint. They discuss coherence, normativity, logic, and probability from this viewpoint.
• Poletiek
  • Poletiek proposes an alternative falsification test to the logical falsification theory of testing. Conversely to logical falsification theory, the Severity of Test is an explanation that involves confirming evidence, instead of falsifying.

• Politzer and Bonnefon
  • They agree with O&C that human reasoning cannot be purely based on logic. However, they have qualms with BR because it doesn’t address how conclusions are formed. Additionally, they believe that O&C ignore the  importance of defining uncertainty.
• Schroyens
  • They challenge the normativeness of BR by bringing up the fact that a rational analysis consists largely on individuals’ differing environments and goals as influences on their rationality. Furthermore, Schroyens believes that it is misleading when O&C ignore algorithmic-level specifications when comparing probabilistic and nonprobabilistic theories.
• Stenning and van Lambalgen
  • They do not agree with O&C’s claim that logical methods cannot encompass nonmonotonicity, and therefore a probabilistic approach is required. They give examples of where BR fails to account for some forms of nonmonotonicity, and further suggest that a non-Bayesian theory must be used in addition.
• Straubinger, Cokely, Stevens
  • While O&C solely address adult reasoning, S/C/S approach reasoning as something that varies over an individual’s lifespan. Because of the variations between individuals and age groups, S/C/S believe that one model (BR) isn’t sufficient to describe human reasoning as a whole.
• Wagenmakers
  • Wagenmakers agrees that the information gain model is the best model to describe the Wason card task. However, he questions why participants don’t select all four cards given the information gain model. He also wonders if incentive, like money, would change the results.

Author’s Response:

R2.1:

O&C denounce Evans’ Dual Process view because it seems possible that System 1 and System 2 could contradict one another. Additionally, they claim that addressing individual differences in reasoning isn’t necessary for determining whether there is a single or multiple human reasoning systems.

R2.2:

The authors observe that deductive reasoning, which is certain, is not observed outside mathematics and thus, their account of reasoning, involves making pragmatic choices due to uncertainty. Thus, rather than working from the “premises alone”, BR allows for “uncertain, knowledge rich” inference.

R2.3:

O&C counter Politzer & Bonnefon’s criticism by providing examples (algorithms and constraint satisfaction neural network implementation of the probabilistic approach) of how BR accounts for the generation of conclusions.

R3.1:

O&C respond to Pfeifer & Kleiter’s statement, that the probability theory inherently includes classic logic, by saying a Bayesian inductive perspective is necessary because classic logic isn’t very applicable to everyday life.

R3.2:

O&C claim that adding a condition of relevance doesn’t address the uncertainty problems because elements outside of mathematics are inherently uncertain. Furthermore, O’Brien’s system doesn’t correctly capture the intuitions of relevance between antecedent and consequence.

R3.3:

The authors argue that resolving clashes between premises can only be obtained by differentiating between stronger and weaker arguments, and degrees of confidence in the premises of those arguments.

They posit that logical methods provide no natural methods for expressing such matters of degree; but dealing with degrees of belief and strength of evidence is the primary business of probability theory.

R3.4:

O&C respond to objections regarding the generalization of probabilistic reasoning and existing conflicts between prior beliefs and logical reasoning. They posit that the “description of behavior in logical or probabilistic terms doesn’t mean that the behavior is governed by logical and probabilistic processes”. The conclude that without probabilistic reasoning, logic cannot accurately capture human patterns of thought

R3.5:

O&C justify the Bayesian approach as a “pragmatic” choice given its wide application in the cognitive and brain sciences. They also assert that the Bayesian assumptions may be “too weak” insofar as it imposes “minimal coherence criteria” on beliefs. Lastly, they dismiss objections regarding justification as they propose probability as a “better” (not the best) means of dealing with uncertainty.

R3.6:

O&C respond to concerns regarding the “rigidity” and “uncertainty” of Bayesian probability. First, they assert that BR doesn’t need to account for all uncertainty, regarding conditionals, as some uncertainty isn’t relevant to the data. Second, they explain that the apparent lack of “rigidity” is Bayesian as it accounts for “pragmatic utterances”. Lastly, they disagree that people can reason deductively about probability intervals as new information is always incorporated from world knowledge.

R3.7:

The authors posit “disinterested” and “goal oriented” methods of inquiry. The former aims to maximize the expected amount of information gotten from a task while the latter maximizes the expected utility of getting information. By adopting a “goal oriented” method, they avoid the postulation of “specific machinery”.

R4: (comprehensive)

The authors criticize “algorithmic” models (eg connectionist models) insofar as they shed no light regarding “why” the modeled processes function. They also argue that “ecological rationality” supplements normative rationality and  “rational analysis aims to explain ecological rationality”. Moreover, they posit that rational analysis is “goal specific” insofar as “rational” refers to information processing systems.

They also  acknowledge the challenges faced when attempting to implement BR on an algorithmic level. Notwithstanding, they assert that “understanding the rational solution to problems faced by the cognitive system crucially assists with explanation in terms of representations and algorithms”. Thus, rational analysis assists algorithmic explanation.

Also, O&C acknowledge that rational analysis may be challenged when there are many, near optimal, rational solutions and, sometimes, finding exactly the optimal solution may be over-restrictive. In such cases, they suggest rational analysis will select a solution based on its “relative goodness”. They also justify the simplicity of the naive Bayes model as it can be justified by Bayesian reasoning.

R5:

First, the authors reveal a doxastic and factual distinction insofar as changing degrees of belief doesn’t entail a change in the real conditional probability. Also, the authors respond to objections regarding the BR model. Most importantly, they state that the experiments were performed “pragmatically” insofar as “it conforms to the current demand of most journals”.  

Discussion Questions:

1. Many commentators feel that BR doesn’t provide an adequate explanation for how people generate conclusions. Could fast and frugal heuristics serve as an explanation? In other words, to what extent do fast and frugal heuristics serve as the “specific machinery” for probabilistic reasoning?
2. Is BR normative or descriptive? Are there any tensions between rational analysis and ecological rationality insofar as the former seems normative and the latter accounts for individual differences.
3. Why is it that people seem more rational in the real world than in the laboratory? That is, why are there more violations of logic when in a controlled setting?

Oaksford and Chater challenge the logicist conception of human rationality. In its place, they advocate Bayesian rationality, which offers a framework for reasoning in the face of uncertainty. Bayesian rationality involves probabilistic reasoning. Here, probability describes an agent’s degrees of belief and is thus considered qualitative/subjective, not numerical.

1. Logic and the Western conception of mind

Oaksford and Chater describe an early approach to rationality, known as the logicist conception of the mind, according to which inferential relations maintain absolute certainty. Oaksford and Chater proceed to use syllogisms in demonstrating that logical arguments are truth preserving: To believe the premises of a logical argument is to believe its conclusion. So, denial of that conclusion is incoherent. O&C note that logical certainty prevents the addition of contingent facts (70). They then introduce two contemporary theoretical accounts of human reasoning—the mental logic and mental models theories. The mental logic view assumes reasoning involves logical calculations over symbolic representations; the mental models view takes reasoning to involve concrete representation of situations (71). In concluding this section, O&C introduce Bayesian rationality as the approach that best deals with the discovery of theory-refuting data.
2. Rationality and rational analysis

This section seeks to demonstrate why a Bayesian perspective is better than a logical one. O&C begin by outlining the six steps of rational analysis (71–72), which includes normative theory in its larger account of empirical data concerning thought and behavior. Rational analysis aims to understand the structure of the problems facing the cognitive system and takes into account the relevant environmental and processing constraints. O&C address two caveats of Bayesian rationality—that rational analysis isn’t a theory of psychological processes and that it doesn’t measure performance on probabilistic or logical tasks (72). The authors assure us, though, that these caveats are not to be of any inconvenience.

3. Reasoning in the real world: How much deduction is there?

This section challenges the use of the logic calculus in everyday reasoning. In reasoning about the everyday world, we usually have only bits of knowledge, some of which we believe only partially and/or temporarily. Moreover, the non-monotonicity of commonsense reasoning means that we can overturn virtually any conclusion upon learning additional information (72). Non-monotonic inferences cannot be accounted for by conventional logic. Importantly, classical logic fails to deal with the notorious “frame problem,” which refers to the difficulty of representing the effects of an action without having to enumerate obvious “non-effects” (73). BR, by creating non-monotonic logics, thus addresses this “mismatch” between logic-based and commonsense reasoning.

4. The probabilistic turn

O&C offer probabilism as an approach that best deals with the uncertainty of everyday reasoning. The authors provide the example of court cases decided by jury. In these situations, new pieces of evidence can modify one’s degree of belief regarding the guilt of the defendant. Here, probability is determined subjectively (74). In the remaining paragraphs, O&C present a modified version of the familiar conditional If A then B. This version, embraced within the cognitive sciences, takes B to be probable, if A is true (74). The authors conclude this section by noting the shift towards probability theory across a number of domains, including philosophy of science, AI, and cognitive psychology.

5. Does the exception prove the prove the rule? How people reason with conditionals

This section deals with the first of three core areas of human reasoning—conditional inference. The authors identify four conditional inference patterns (Fig. 1): (1) Modus Ponens, (2) Modus Tollens, (3) Denying the Antecedent, and (4) Affirming the Consequent. Of the four, two are logical fallacies—Denying the Antecedent and Affirming the Consequent.

Figure 2, Panel A presents data from experiments that asked people if they endorse each of the four inference patterns. The observed results diverge from the predictions of the standard logical model. Logicists attempt to account for the divergence by allowing people the pragmatic biconditional interpretation, though it is logically invalid (75).

The Bayesian approach, however, appeals only to probability theory. The probabilistic account of conditional inference entails four key ideas (75):

1. P(if p then q) = P(q|p), aka “The Equation;”
2. Probabilities are interpreted as degrees of belief, and this allows for belief updating;
3. The Ramsey Test determines conditional probabilities; and
4. By conditionalization (i.e., when the categorical premise is certain, not supposed), our new degree of belief in q is equal to our prior degree of belief in p→q. Quantitatively: If P0(q|p) = 0.9, and P1(p) = 1, then P1(q) = 0.9. The takeaway here is that, from a probabilistic account, we are able to update our degrees of belief in q upon learning p is true without making too strong of a claim.

The remainder of this section deals with biases observed in conditional inference. The first of these are the inferential asymmetries (that MP is more widely endorsed than MT and that AC is more widely endorsed than DA). Though the probabilistic account can explain inferential asymmetries without invoking pragmatic inferences or cognitive limitations, it distorts the magnitudes of the MP–MT and DA–AC asymmetries (Panel C). Learning that the categorical premise is true can alter one’s degree of belief in the conditional, and this constitutes a violation of the rigidity condition. This violation lowers our degree of belief in P0(q|p), and this lower estimate, when included in calculations of probabilities for DA, AC, and MT, in turn explains the resulting magnitudes of the asymmetries (76).

The second of these biases is the negative conclusion bias—that people endorse DA, AC, and MT more often when the conclusion contains a negation. Since the probability of an object being red, for instance, is lower than the probability of it not being red, P0(p) and P0(q) take on higher values when p or q is negated. So, a seemingly irrational negative conclusion is simply attributed to a “high probability conclusion effect” (77).

The authors conclude the section with an overview of a small-scale implementation of the Ramsey test and question whether future implementations can explain the full range of empirical observations in conditional inference.

6. Being economical with the evidence: Collecting data and testing hypotheses

This section deals with the second of three core areas of human reasoning—data selection. Recall the Wason selection task. In testing the hypothesis if there is an A on one side of a card, there there is a 2 on the other, one should seek out falsifying examples (i.e., p, not q cases). Accordingly, one should select the A and 7 cards. This does not seem to be the case, however: Participants more often select cases that confirm the conditional (confirmation bias) (77).

Bayesian hypothesis testing is comparative—not falsifying. The optimal data selection (ODS) model assumes that people compare the dependence hypothesis (HD)—that P(q|p) is higher than the base rate of q—with an independence hypothesis (HI), according to which P(q|p) is the same as the base rate of q (78). Initially, people are thought to be equally uncertain about which hypothesis is true. The goal of the selection task, then, is to reduce this uncertainty. Using Bayes’ theorem (see Note 2), one can calculate her new degree of uncertainty about HD upon discovering p→q.

The ODS model is based on information gain, and participants in Wason’s task base their decisions on expected information gain. The ODS model also assumes the rarity of the properties that belong to the antecedent and consequent. So, the expected informativeness of the q card is greater than that of the not q card since almost certainly we would learn nothing about our hypothesis if we investigated not q cases. This approach is at odds with the falsification perspective but agrees with the empirical data (78). The ODS model thus suggests that performance on Wason’s task is in fact consistent with rational hypothesis testing behavior.

The authors also address the apparently non-rational matching bias, as a result of which participants match values named in the conditional. Given the rule if A then not 2, people tend to make the falsifying response: They select the A and 2 cards (rather than the 2 and 7 cards). Because of the rarity assumption, however, not q is a high probability category, and a high probability consequent thus warrants the falsifying response (79).

In the remainder of the section, the authors discuss deontic selection tasks (which involve conditionals that express rules of conduct, not facts about the world). In such instances, people do select the “logical” cards (the p and not q cards). Here, it is not a hypothesis that is tested but a regulation; it is useless to confirm or disconfirm how people should act. Rather, participants seek out violators of the rule (79). Moreover, in such selection tasks, people select cards to maximize expected utility, and because only the p and not q cards have positive utilities, these are the cards chosen (80). This model has also been extended to rules with emotional content.

7. An uncertain quantity: How people reason with syllogisms

This section deals with the last of three core areas of human reasoning—syllogistic inference, which relates two quantified premises, of which there are four types: all, some, some…not, and none. Of the 64 possible syllogisms, 22 are logically valid (Table 1).

The Probabilistic Heuristics Model (PHM) employs the probabilistic approach to syllogisms. PHM’s most important feature is that it also applies to generalized quantifiers, like most and few (82). PHM assigns probabilistic meanings to terms of quantified statements (81). For instance, the meaning of the universally quantified statement All P are Q can be given as P(Q|P) = 1. Similarly, the generally quantified statement Most P are Q can be understood as 0.8 < P(Q|P) < 1, for instance.

These interpretations are then used to build simple dependency models of quantified premises, and these models can be parameterized to determine which inferences are probabilistically valid (81).

The PHM model also assumes that, in general, because the probabilistic problems encountered by the cognitive system are very complex, people employ simple and effective heuristics to reach “good enough” probabilistic solutions (83).

There are two background ideas to keep in mind regarding heuristics (81): (1) the informativeness of a quantified claim and (2) the probabilistic entailment between quantified statements. A claim is informative in proportion to how surprising (unlikely) it is. No P are Q, which is very likely to be true, is thus an uninformative statement; All P are Q is the most informative. Regarding the second idea, the quantifier All probabilistically entails (p-entails) Some; Some and Some…not are mutually p-entailing.

There are two types of heuristics for syllogistic reasoning (82)—the generate heuristics (produce candidate conclusions) and the test heuristics (evaluate the plausibility of candidate conclusions).

There are three generate heuristics: (G1) the min-heuristic, (G2) p-entailments, and (G3) the attachment-heuristic.

The two test heuristics are (T1) the max-heuristic and (T2) the some_not-heuristic. In general, where there is a probabilistically valid conclusion, these heuristics identify it successfully. The authors offer experimental data in support of this claim.

8. Conclusion

Taken together, the empirical data provided support O&C’s probabilistic approach to human rationality. In sum, the cognitive system is best understood as building qualitative probabilistic models of the uncertain world.

Questions:

1. Does Bayesian rationality actually account for the uncertainty of everyday situations that logical methods ignore?
2. Can logical rationality and Bayesian rationality exist in unison?
3. If classical logic is indeed inadequate in its explanation of human rationality, what becomes of the normative/descriptive gap?
4. How might a program like ECHO accommodate the Bayesian perspective, if at all?

Stanovich and West set out to address two major topics in their response: the role of individual differences in the normative/descriptive gap and the two-process model of evolutionary rationality (System 1) and normative rationality (System 2).

R1. Individual differences and the normative/descriptive gap

R1.1. Normative applications versus normative models.

First, they touch on the distinction between normative applications and normative models, claiming that many authors misunderstood their attempt to use patterns of individual differences. S&W clarify that they used patterns of individual differences not to determine whether normative models are adequate, but to determine when to apply a certain normative model to a particular situation. They stress the importance of empirical data in determining the applicability of certain normative models (rather than their correctness). With their data, S&W were trying to shed light on specific norm applications in particular situations and whether the norms applied were appropriate to the situation or not (for example, they reference the cabs problem and the applicability of Bayes’ Theorem to the situation).

R1.2. The understanding/acceptance assumption.

They also discuss the understanding/acceptance assumption and its efficacy as a tool for judging different explanations for the normative/descriptive gap. They confirm Schneider’s view that the assumption is “necessary but not sufficient criterion” (702) for understanding the gap in behavior. Notably, the assumption posits that intelligent people are more likely to apply the correct normative application. S&W next turn to the Allais paradox, countering the belief of Ayton and Hardman that the assumption fails if it cannot judge the normative model applied in this case. However, S&W argue that norm misapplication is at fault for the dispute arising from this paradox, and the understanding/acceptance assumption should not be held accountable.

R1.3. Relativism and rational task construal.

In the next section, S&W look at rational task construal. S&W cite the Panglossian view on rational task construal, that any and all task construals are rational and intransitivity can be eliminated by construing the problem in such a way that removes the intransitivity. In this view, the task is construed in such a way that makes the task and task responses rational and the person involved rationally competent. However, alternative task construals do not protect one from irrationality, as one can be charged with irrationality in a different part of the process. Cognitive evaluation is crucial for determining the rationality of task construals.

R1.4. Cognitive ability and the SAT.

S&W strongly defend their usage of tasks from the heuristics and biases literature and SAT scores as different measures of cognitive ability. They claim that the tasks they measured were of a completely different nature than those on the SATs, and that, notably, reasoning tasks do not have a correct answer. Correlation, or lack of correlation can be used to examine the normative model and task construal used in particular problems. S&W refute the claims of those who criticize their use of SAT performance as an indicator of cognitive ability. They draw clear relationships between general intelligence, working memory, computational capacity, and fluid intelligence, all reflected by SAT scores. Notably, the authors mention that education does not appear to affect performance on heuristics and biases tasks. They also note their usage of converging measures of cognitive ability that measured intelligence in a different way than the SATs and showed the same correlations.

R1.5. Alternative interpretations of computational limitations

In their target article, S&W interpreted cognitive ability measures as agents of overall cognitive intelligence. They admit, however, that there are several alternatives: (1) Cognitive ability measures may be indicators of an individual’s tendency to respond to problems with appropriate strategies, and (2) that cognitive ability may portray the number of different problem representations with which an individual can cope. These alternative interpretations therefore lead us to three distinct computational limitations: (1) limitations in how one can successfully process information, (2) limitations in the flexible deployment of strategies to solve a problem, and (3) limitations in the types of problem representations one can handle and, thus, the types of strategies one can use to solve a problem.

R1.6. Alternative Construals as a Computational Escape Hatch.

Ball & Quayle bring up an interesting idea of a computational escape hatch, which prompt S&W to blur the line between seemingly distinct notions of alternative task construal and computational limitations.  Alder suggests, and S&W agree, that perhaps tasks are being interpreted as different from what the experimenter intended because individuals cannot fully grasp that task, and thus the normative model. S&W mention that alternative construals can be used as computational escape hatches by either being consciously aware of all the alternative task interpretations and choosing the one with the lowest computational demands, or choosing it without being aware of the other alternatives.

R1.7. Thinking Dispositions and Cognitive Style.

S&W agree with Kuhberger and argue that thinking dispositions and cognitive capacities are different, distinguishable concepts as they function on different levels of analysis. They argue that cognitive abilities indicate individual differences in the efficiency of processing at the algorithmic level while thinking dispositions indicate individual differences at the intentional level. S&W have found that thinking dispositions can explain variance independent of cognitive capacity supporting the separability of the two.

R2. Rationality and Dual Process Models

As Friedrich and others point out, S&W are proud of their ability to “bridge between the evolutionary and traditional biases perspectives” (707), especially in order to reinforce their argument for the dual-system processing. The goals of the 2 systems should be similar, to strive for normative rationality, but Frisch stresses that it is not necessarily that both systems will compute and conclude the same way. System 1 processes, or lack of processing, are often attributed to any irrational behavior. However, S&W recognize the potential downfall in overanalytics, citing Hardman’s research in which they found those who were analytical made less effective decisions. They also continue to support the rationality behind the Meliorist framework.

R2.1. Short Leash, Long Leash Goals

S&W explain short leash vs long leash using the Mars Rover example, in which something that can no longer be “short leash” controlled remotely must be given the “long leash” ability to control itself (710). Giving Systems 1 short leash goals of “if A, then B” allows it to remain functionally rational. System 2 is more equipped to reach for long leash goals, in which we boil it down to “Do whatever you think is best,” as offered by Dawkins. In order for System 2 to correctly strive for this goal, however, it must be given the tools to analyze and also be able to recognize what “is best,” a point that Greene & Levy comment on. S&W here push against Ayton’s notion that a rational bee with long leash goals should not sacrifice itself for the good of the hive, saying that evolutionary psychologists mistakenly presuppose that those with evolutionary rationality necessarily have individual rationality. System 2, unlike System 1, is capable of continuous goal evaluation.

R2.2. Cultural Evolution of Normative Standards

Schneider believes that “cultural evolution of norms somehow present difficulties for our conceptualization” (712). S&W disagree, as they feel cultural history of evolution supports that individuals creating progressive change in standards are of high intelligence. Then, others with less intelligence can use the newly developed standards themselves as learners. Panglossians often downplay the evolution of reasoning norms, since an “incorrect” response should never happen, but fail to recognize that changing norms can allow this once “incorrect” response to now be “correct.”

R3. Process and Rationality

Commentators such as Hoffrage, Kahneman, and Reyna critic the lack of algorithmic-level process models for many of the tasks mentioned in the target article. While S&W agree that these process models are important, they argue that this was not the point of their research program. They felt that rather than focusing entirely on the algorithmic-level model it was best to explore intentional-level models and its variance in rationality as well. They argue that exploring intentional-level constructs does not detract from the search for more extensive algorithmic-level specification and that the two levels may actually have a synergistic interplay.

R4. Performance Errors

Hoffrage treats many errors that Stanovich & West as computational as performance errors, most notably “recurring motivational and attentional problems” (713). However they argue that this is not appropriate because such errors are “in fact like cognitive styles and thinking dispositions at the intentional level,” (713) since stability and predictability goes against the random nature of performance errors. Stanovich and West argues that this is the most important implication of a performance error – that it is to be considered “trivial”, and reminds us that it only becomes something significant when it is repeated and forms a pattern.

R5. The Fundamental Computational Bias and “Real Life”

Commentators criticise the authors for focusing on problems that are not similar to real life, however the authors respond by arguing that “real life” is no longer real life as technology has “presented evolutionarily adapted mechanisms with problems they were not designed to have.” (714) They gave examples such as the food people eat as well as communications and advertisements. Commentators such as Ayton and Hardman therefore point out the necessity of the “fast and frugal” heuristics studied by Gigerenzer. Kahneman acknowledges that this process is important, but reminds readers that system 2 is still necessary to correct the associated biases. The authors seem to agree with these analyses and do think that most humans still live in the world of system 1 and laments that there are “very few situations where System ½ mismatches are likely to…have knock-on effects.” (714)

R6. Potentially Productive Ideas

Stenning & Monaghan suggests other ways to reparse the System 1/2 differences, such as cooperative vs. adversarial communication or explicit vs. implicit knowledge. Moshman’s dissection between “what a system computes,” and “how the processing is implemented,” may lead to greater explanatory power with greater clarification. The authors acknowledges other ideas such as a finer-grained scoring system by Hoffrage & Okasha, taking into account the test subjects decision-making history by Fantino, and the inclusion of further tools such as developmental data by Reyna & Klaczynski or the notions of rationality in other literatures such as philosophy by Kahneman.

R7. Complementary Strengths and Weaknesses

The different “camps” have all advanced the field through their own lenses respectively; the Panglossians have demonstrated that the answers to real-world problems are often “in human cognition itself,” (717) and that mankind is already optimising in the real world. The only necessary action is to characterise and optimise further the process. The Apologists have shed light on the power of evolution and its ability to shape cognition. The Meliorists have argued for the possibility of cognitive change and warned against the possible results of mismatches between the ways we think and the way we should think in a modern day society.

Similarly each have its weaknesses as Meliorists sometimes jump ship too quickly and blame flaws in reasoning while the Panglossians are often forced into uncomfortable positions to defend the human rationality; thus it is necessary to be open to the other possibility and take that into account. The Apologists can sometimes be too old-fashioned and fail to recognise the huge differences between the modern society and that during which humans evolved.

Discussion Questions:

1. Do you think the characterizations by Stanovich & West in R7 are valid?
2. Did S&W defend their choice to use the SATs as a cognitive ability measure well?
3. Do you agree with the criticism that the alternative task construal is a way to mask computational limitations?
4. Does S&W argument that today’s society is too technologically advanced for System 1 to continue to adapt make sense?

Stanovich and West (S&W) attempt to maintain the rationality of human thought by explaining the gaps between descriptive and normative models for decision making. Often times humans’ patterns of judgment do not follow the normative models of decision-making and rational judgment. There are two major schools of thought in regards to how these inconsistencies should be accounted for: the Meliorists and the Panglossians. Meliorists believe that there is a deficiency in human cognition and that we should reevaluate and reform our way of thought to coincide with the normative model. Panglossians believe that these gaps are not a result of human irrationality and that the normative model should be changed rather than our way of thinking.

Rather than clearly identifying themselves Meliorists or Panglossians S&W aim to lay out the possible explanations for the differences between human responses and normative performance. They attribute these deviations from the normative to four possibilities: 1) performance error, 2) computational limitations, 3) application of the wrong normative model and, 4) alternative task construal, all of which maintain the rationality of human reasoning.

Performance Error

Performance error is the unsystematic deviation from the normative model, or essentially random lapses in judgment due to either distraction, lack of attention, temporary memory deactivation, etc. It is these momentary attention, memory, or processing lapses that cause the individual variation in judgment. Performance error as an explanation can be taken to its farthest limits by attributing all deviations from the normative model to performance error.

Computational Limitations

Computational limitations, unlike performance error, are systematic deviations from the normative model as a result of deficiencies in human cognition. The individual differences in human decision-making result from different levels of computational limitation (some people have more brain power than others). Variations in cognitive ability lead to variation in human judgment.

Application of the wrong normative model

The process of assigning a normative model to a problem is a complex and involved action and it is extremely difficult to find a problem that can be clearly matched with a normative model. As a result of this complexity there is a lot of room for error in decision-making and possible application of the wrong normative model, leading to individual variation especially if everyone cannot identify and then use the same models for judgment.

S&W identify a correlation between the understanding and the acceptance of a normative model, otherwise known as the understanding/acceptance principle. The greater the understanding of a model is, the higher the acceptance of that model. With more understanding an individual is driven towards to normative model, making individuals with higher cognitive ability more likely to fall in line with normative thought. It is not always clear which pattern of reasoning should be used for specific decision making situations. Higher understanding decreases the possibility of applying the wrong normative model by increasing one’s ability for reflection and evaluation of an issue, thereby increasing one’s ability to identify the appropriate response.

Alternative task construal

Alternative task construal attributes individual variation in reasoning to the subject having a different interpretation of a task than the experimenter intended, leading the subject to provide the appropriate normative answer to a different problem, unlike with application of the wrong normative model. Application of the wrong normative model focuses on individuals’ inability to identify the correct way to solve a problem, while the variation from alternative task construal comes from individuals interpreting different problems all together.

Given that normative reasoning is being used as a standard for decision-making and because alternative task construal is a source of variation among individuals’ reasoning, we have to evaluate which interpretations/construals are appropriate or inappropriate for that standard. The possibility for various interpretations implies that we now need principles for rational construal. Finding these principles can be done using the same methods of justification used in instrumental rationality. For example, a construal could be deemed appropriate or inappropriate based on how efficient they are at helping a subject reach their goals.

S&W propose the dual process theory as one possible explanation for why we interpret tasks differently. System 1 processing is a system of intelligence focused on the maximization of advantages based on the evolutionary interests of the human race and is largely unconscious. System 2 processing is a system of intelligence focused on the maximization of individual advantages based on the interests of the whole person. The two systems tend to yield different construals and cause variation in individual reasoning because not everyone has the cognitive ability to employ system 2 processing in the same manner.

Conclusion

S&W after laying out the four possible sources of variation between individual reasoning, conclude that the most compelling explanations for the sources of decision-making differences are application of the wrong normative model and alternative task construal. Ultimately, S&W agree with the Meliorist conclusion that there are individuals differences that do not fit in with the normative model of decision-making and human reasoning is not ideal “as is”.

Responses:

Note: We will focus on the responses up to pg 680 with Hunt.

One of the biggest points of concern for a lot of the commentaries we encounter in this section is S&W’s claim that failure to use a certain normative model in a given situation is indicative of human irrationality. Ayton argues that trying to determine what is “normative” is not useful because decisions cannot be evaluated so easily. Bees, for example, violate what we expect to be the normative model, but their behavior is still successful if they survive. So although bees do not follow the normative model, they are not considered “irrational.” Baron states that normative models should not be justified by consensus/intuitive judgements, but should instead come from analysis of specific situations that are supported by more evidence. Goodie & Williams question defining reasoning in terms of aptitude, because aptitude must be measured using reasoning and this is circular.

Others are compelled by the way S&W outline the dual-processing model, but think that it could be better developed. Ball & Quayle provide an especially interesting view by further explaining different scenarios in which the systems are used, highlighting that System 1 is often used as an escape hatch for when System 2 is overloaded. Friedrich takes issue with the distinction between the two systems, especially the overemphasis on System 2 that comes from the experimenter’s expectations and from the fact that most study samples come from elite universities where analytic processing is emphasized. Frisch offers that instead of trying to figure out which system is “better” or used more often, we should view the systems as working in tandem (like yin and yang) – and that a careful balance between the two would be most effective.

Finally, some commentators question how S&W’s system would actually function in the “real world.” DeKay et al and Greene & Levy argue that S&W need to consider the evolutionary aspect of our psychological mechanisms so that we can better understand our behavior and more clearly define what is normative. Using the evolutionary lens explains that sometimes errors exist because they were evolutionarily advantageous in the past, and that variation is important to allow us to adapt to new and unpredictable environments. Girotto also explains that people in “real world” scenarios might choose non-normative responses in order to choose a response that would be optimal for a group (ie, a collectively optimal decision).  Hardman questions the viability of the understanding/acceptance principle and provides examples of when that has not actually been true (sometimes people do not pick the option they know more about).

Discussion Questions:

1. Of the four explanations that S&W provide for deviations from normative models, which do you think is most compelling?
2. Are we satisfied by S&W’s use of the SAT as a way to measure intelligence? What would be a better way to account for different types of intelligence?
3. Of the various critiques to S&W, which do you think is most problematic for them?
4. S&W ultimately seem to favor the Meliorist perspective. Do you agree with this conclusion, or do you find the Panglossian view more accurate?