qoder-config/skills/deep-research/references/logical_fallacies.md

Logical Fallacies Catalog — 30+ Fallacies for Research Review

Purpose

Reference catalog of logical fallacies commonly encountered in research. Used by the devils_advocate_agent.

Formal Fallacies (Invalid Logical Structure)

1. Affirming the Consequent

Structure: If P then Q; Q is true; therefore P is true. Example: "If a university has high research funding, it has good outcomes. This university has good outcomes. Therefore, it must have high research funding." Problem: Q can have multiple causes.

2. Denying the Antecedent

Structure: If P then Q; not P; therefore not Q. Example: "If enrollment increases, revenue increases. Enrollment didn't increase. Therefore, revenue didn't increase." Problem: Revenue can increase from other sources.

3. Undistributed Middle

Structure: All A are B; All C are B; therefore All A are C. Example: "All successful programs use technology. Our program uses technology. Therefore, our program is successful." Problem: B (technology use) is shared but doesn't link A and C.

4. False Dilemma / False Dichotomy

Structure: Either A or B; not A; therefore B. Example: "Either we adopt online learning completely or maintain traditional methods." Problem: Many hybrid options exist.

Informal Fallacies

Relevance Fallacies

5. Ad Hominem

Description: Attacking the person rather than the argument. Research Example: "This study's conclusions are unreliable because the author works for a for-profit university." Correct Approach: Evaluate the methodology and evidence, not the author's affiliation (though COI should be noted).

6. Appeal to Authority

Description: Accepting a claim solely because an authority figure endorses it. Research Example: "Published in Nature, so the findings must be valid." Correct Approach: Even prestigious journals publish flawed studies. Evaluate on merit.

7. Appeal to Tradition

Description: Arguing something is correct because it has always been done that way. Research Example: "This metric has been used for 30 years, so it must be the best measure." Correct Approach: Evaluate whether the metric is still valid in current context.

8. Appeal to Novelty

Description: Arguing something is better because it's new. Research Example: "This new framework must be superior to the established one." Correct Approach: Novelty doesn't equal improvement. Compare on evidence.

9. Appeal to Popularity (Bandwagon)

Description: Arguing something is true because many people believe it. Research Example: "Most researchers in the field use this method, so it must be the best." Correct Approach: Popularity doesn't validate methodology. Assess independently.

10. Red Herring

Description: Introducing an irrelevant topic to divert from the argument. Research Example: Responding to criticism of methodology by discussing the importance of the topic.

Evidence Fallacies

11. Cherry-Picking (Selection Bias)

Description: Selecting evidence that supports the conclusion while ignoring contradictory evidence. Research Example: Citing 5 studies that support the hypothesis while omitting 12 that don't. Detection: Compare cited sources against comprehensive search results.

12. Confirmation Bias

Description: Seeking, interpreting, and remembering information that confirms pre-existing beliefs. Research Example: Designing search terms that are more likely to return supportive results. Detection: Check if search strategy was neutral; look for actively sought disconfirming evidence.

13. Survivorship Bias

Description: Drawing conclusions only from "survivors" (successes), ignoring those that didn't survive. Research Example: "All top-ranked universities implemented X" — ignoring universities that implemented X and didn't improve. Detection: Ask "what about those that failed?"

14. Anecdotal Evidence

Description: Using individual stories as proof of a general claim. Research Example: "One university tripled enrollment after rebranding, so rebranding drives enrollment." Detection: Is this a systematic finding or an isolated case?

15. Hasty Generalization

Description: Drawing broad conclusions from insufficient evidence. Research Example: "Three case studies from Taiwan show X, therefore this applies to all Asian universities." Detection: Is the sample representative? Is the generalization proportionate to the evidence?

Causal Fallacies

16. Post Hoc Ergo Propter Hoc

Description: Assuming that because B followed A, A caused B. Research Example: "After implementing the new curriculum, graduation rates improved. Therefore the curriculum caused the improvement." Detection: Were there confounders? Was there a control group?

17. Cum Hoc Ergo Propter Hoc (Correlation ≠ Causation)

Description: Assuming that correlation implies causation. Research Example: "Universities with more international students have higher rankings, so international students cause higher rankings." Detection: Is there a plausible mechanism? Could both be caused by a third factor?

18. Reverse Causation

Description: Getting cause and effect backwards. Research Example: "Good facilities attract students" when actually "student fees fund better facilities." Detection: Consider temporal order and alternative causal directions.

19. Ecological Fallacy

Description: Inferring individual-level conclusions from group-level data. Research Example: "Countries with more education spending have higher GDP, so spending on education makes individuals richer." Detection: Are individual-level and group-level relationships the same?

20. Simpson's Paradox

Description: A trend present in subgroups reverses when groups are combined. Research Example: Department A and B both show improving retention, but the university overall shows declining retention (due to shifting enrollment proportions). Detection: Always check disaggregated data alongside aggregate.

Reasoning Fallacies

21. Straw Man

Description: Misrepresenting an opponent's argument to make it easier to attack. Research Example: Critic says "this method has limitations" → Author responds "my critic says the entire study is worthless." Detection: Does the refutation address the actual criticism?

22. Moving the Goalposts

Description: Changing the criteria for success after seeing results. Research Example: Defining "program success" as enrollment growth, then shifting to "student satisfaction" when enrollment drops. Detection: Were success criteria pre-defined?

23. Slippery Slope

Description: Arguing that one action will inevitably lead to an extreme outcome. Research Example: "If we allow flexible admission criteria, academic standards will collapse entirely." Detection: Is each step in the chain actually probable?

24. Circular Reasoning (Begging the Question)

Description: The conclusion is assumed in the premise. Research Example: "This university is excellent because it is highly ranked, and it is highly ranked because it is excellent." Detection: Does the argument depend on the truth of what it's trying to prove?

25. No True Scotsman

Description: Redefining a category to exclude counterexamples. Research Example: "All quality assurance systems improve outcomes." "But system X didn't." "Well, X wasn't a true quality assurance system." Detection: Is the definition being modified to fit the claim?

26. Equivocation

Description: Using a term in two different senses within the same argument. Research Example: "Quality" used sometimes to mean "standards compliance" and sometimes to mean "student satisfaction." Detection: Is the key term defined consistently throughout?

Statistical Fallacies

27. Base Rate Neglect

Description: Ignoring the base rate (overall probability) in favor of specific information. Research Example: "This program has a 90% satisfaction rate" — but the base rate for all programs is 88%. Detection: Always compare against relevant base rates.

28. Regression to the Mean

Description: Extreme performances naturally tend back toward average on subsequent measurements. Research Example: "Our intervention improved scores for the lowest-performing students" — they may have improved anyway. Detection: Was there a control group? Were initial measurements extreme?

29. Texas Sharpshooter

Description: Finding a pattern in random data by focusing on clusters and ignoring misses. Research Example: Running 20 statistical tests and reporting only the 1 that was significant. Detection: Were hypotheses pre-registered? Was multiple testing corrected for?

30. Gambler's Fallacy

Description: Believing past random events influence future random events. Research Example: "This institution has declined for 5 years, so it's due for improvement." Detection: Is there a causal mechanism for reversal, or is this just pattern-seeking?

31. McNamara Fallacy (Quantitative Bias)

Description: Making decisions based solely on quantitative metrics while ignoring qualitative factors. Research Example: Ranking universities only by publication counts, ignoring teaching quality and community impact. Detection: Are important but hard-to-measure factors being excluded?

32. Goodhart's Law

Description: "When a measure becomes a target, it ceases to be a good measure." Research Example: Universities gaming rankings metrics instead of genuinely improving quality. Detection: Has the metric become a target? Are there signs of metric manipulation?

Quick Reference: Detection Questions

Ask This	Detects
"Does B have other possible causes?"	Post hoc, false cause
"What about the failures?"	Survivorship bias
"Is this sample representative?"	Hasty generalization
"Were criteria defined before results?"	Moving goalposts, Texas sharpshooter
"Is the key term used consistently?"	Equivocation
"What's the base rate?"	Base rate neglect
"What evidence was left out?"	Cherry-picking, confirmation bias
"Is this the actual argument being made?"	Straw man
"Can we distinguish correlation from causation?"	Cum hoc, ecological fallacy
"Are individual and group levels being mixed?"	Ecological fallacy, Simpson's paradox