qoder-config/skills/deep-research/references/preregistration_guide.md

Preregistration Guide — Research Preregistration Guide

Purpose

Decision guide and operational manual for research preregistration. Assists the research_architect_agent in determining whether preregistration is needed during the methodology design stage, and guides researchers through the preregistration process.

---

1. Preregistration Decision Tree

Does your research have the following characteristics?
│
├── Confirmatory research (hypothesis testing)
│   └── Strongly recommend preregistration
│       ├── Has pre-specified statistical hypotheses → Preregister
│       ├── Will conduct significance testing → Preregister
│       └── Has primary outcome variables → Preregister
│
├── Exploratory research
│   └── Preregistration not required (but optional)
│       ├── Qualitative research → Typically not preregistered
│       ├── Data mining / EDA → Typically not preregistered
│       └── But you can preregister the research design and analysis process
│
├── Systematic review / Meta-analysis
│   └── Strongly recommend registration (PROSPERO)
│       └── Many journals require systematic reviews to be pre-registered
│
├── Randomized controlled trial (RCT)
│   └── Must register
│       ├── ICMJE requires RCTs to be pre-registered
│       └── Most journals will not accept unregistered RCTs
│
├── Replication study
│   └── Strongly recommend preregistration
│       └── Preregistration clearly distinguishes original from modified hypotheses
│
└── Secondary data analysis
    └── Recommend preregistration
        └── Prevents HARKing (Hypothesizing After Results are Known)

When Preregistration Is Not Needed

• Purely qualitative research (grounded theory, phenomenology)
• Exploratory data analysis (no pre-specified hypotheses)
• Theoretical or philosophical research
• Literature reviews (except systematic reviews)
• Case reports or case studies

When Preregistration Is Strongly Recommended

• Any research involving hypothesis testing
• Research involving multiple comparisons
• Research needing to distinguish confirmatory vs. exploratory analyses
• Research that may be questioned for p-hacking or HARKing
• When applying for research funding (demonstrates research rigor)
• When journals explicitly require or encourage preregistration

---

2. Preregistration Platform Overview

Platform	Applicable Field	Features	Cost
OSF Registries	All disciplines	Most widely used, multiple templates, DOI, permanent preservation	Free
PROSPERO	Systematic reviews	Dedicated to systematic reviews and meta-analyses	Free
AEA Registry	Economics	American Economic Association's RCT registration platform	Free
AsPredicted	All disciplines	Simplified preregistration (9 questions), quick to complete	Free
ClinicalTrials.gov	Clinical trials	US FDA-required RCT registration	Free
EGAP	Political science	Experiments in Governance and Politics	Free
RIDIE	Development economics	Registry for International Development Impact Evaluations	Free

Platform Selection Guide

What is your research?
│
├── Systematic review / meta-analysis → PROSPERO
├── Clinical trial / medical intervention → ClinicalTrials.gov
├── Economics RCT → AEA Registry
├── Just need simple preregistration → AsPredicted
└── All other research → OSF Registries (recommended)

---

3. 21-Item Core Content Checklist

Based on the OSF Standard Pre-Data Collection Registration format, the following are the 21 core items:

A. Study Information

#	Item	Description
1	Study title	Descriptive title
2	Authors/Research team	All researchers' names and affiliations
3	Research questions	Main research questions (clear, specific)
4	Hypotheses	Pre-specified hypotheses (including directional predictions)

B. Design Plan

#	Item	Description
5	Study design	Experiment/observational, between/within-subjects, factorial design, etc.
6	Randomization	Randomization method (if applicable)
7	Blinding	Blinding level and implementation (if applicable)
8	Conditions/manipulations	Specific description of each experimental condition/group

C. Sampling Plan

#	Item	Description
9	Existing data	Whether existing data is being used; nature and status of data
10	Data collection procedures	How data will be collected (survey, interview, experiment, archival)
11	Sample size	Planned sample size and basis for determination
12	Sample size rationale	Power analysis or other sample size calculation method
13	Stopping rule	When to stop collecting data (fixed N / target power reached / time cutoff)

D. Variables

#	Item	Description
14	Manipulated variables	Operational definition of independent variables
15	Measured variables	Operational definition and measurement instruments of dependent variables
16	Indices	Specific indicators for each variable (scales, items, scoring methods)

E. Analysis Plan

#	Item	Description
17	Statistical models	Primary statistical methods for analysis
18	Transformations	Data transformation plan (e.g., log transformation, standardization)
19	Inference criteria	Significance level (alpha), correction methods, effect size reporting
20	Data exclusion	Exclusion criteria (outlier definition, attention check failure, etc.)
21	Exploratory analyses	Planned but non-primary hypothesis analyses

---

4. Higher Education Research Preregistration Examples

Example: Effect of Teaching Strategy on Learning Outcomes

Title: The Effect of Flipped Classroom on University Students' Critical Thinking
       Skills: A Randomized Controlled Trial

Hypotheses:
H1: Students receiving flipped classroom instruction will score significantly
    higher on the CCTST than students receiving traditional lectures
H2: The benefit of flipped classroom will be greater for students with low
    prior knowledge than for those with high prior knowledge

Design: Cluster-randomized controlled trial (class as randomization unit)
Sample: 12 classes (6 experimental / 6 control), approximately 40 students
        per class, total 480
Power: 80% power to detect d = 0.4, alpha = .05, ICC = 0.05

Primary outcome: CCTST post-test score (controlling for pre-test)
Secondary outcomes: Final exam grade, learning motivation scale
Analysis: Multilevel modeling (students nested in classes)

Exclusion criteria:
- Attendance rate < 50%
- Both pre-test and post-test incomplete
- Attention check questions answered incorrectly

Exploratory analyses:
- Gender × teaching method interaction effect
- Learning motivation as a mediating variable

Example: Systematic Review of University Dropout Factors

Title: Factors Influencing University Student Dropout Decisions in Taiwan:
       A Systematic Literature Review

Research question: What factors influence university student dropout decisions
                   in Taiwan?
Databases: Airiti Library, TSSCI, Scopus, Web of Science
Search strategy: (dropout OR withdrawal OR leave)
                 AND (university OR higher education)
                 AND (Taiwan)
Time range: 2010-2025
Inclusion criteria:
- Studies with Taiwan university students as research subjects
- Explore causes or factors of dropout/withdrawal
- Peer-reviewed journal articles or theses/dissertations
Exclusion criteria:
- Research subjects below high school level
- Pure policy commentary (no empirical data)
Quality assessment: Mixed Methods Appraisal Tool (MMAT)
Synthesis method: Thematic synthesis
Registration platform: PROSPERO

---

5. Preregistration Disclosure Statement Templates

Disclosing Preregistration in a Paper

Standard Statement (Preregistered)

This study was preregistered on [Platform] prior to data collection
(registration number: [NUMBER]; URL: [URL]). All hypotheses, sample size
rationale, and analysis plans were specified before data collection began.
Deviations from the preregistered plan are noted in [section/supplementary
materials].

Disclosure of Deviations from Preregistration

Deviations from preregistered plan:
1. [Deviation description]: [Reason for deviation]
2. [Deviation description]: [Reason for deviation]
These deviations do not affect the confirmatory nature of the primary analyses.
The preregistered analyses are reported as planned; additional exploratory
analyses are clearly labeled.

Disclosure When Not Preregistered

This study was not preregistered. While the hypotheses were formulated before
data analysis, the distinction between confirmatory and exploratory analyses
should be interpreted with this limitation in mind.

---

6. Preregistration vs. Registered Reports

Aspect	Preregistration	Registered Reports
Definition	Research plan publicly registered in advance	Research plan submitted to a journal for pre-review
Review	Does not undergo peer review	Stage 1 peer review (research design)
Acceptance timing	Paper submitted only after completion	Receives "In-Principle Acceptance" (IPA) after passing Stage 1
Results bias	Reduced but not eliminated (researchers can still selectively report)	Substantially eliminated (published regardless of results)
Publication bias	Cannot solve	Effectively solved (null results also published)
Applicable journals	All journals	Only journals accepting Registered Reports
Difficulty	Low (just fill in a form)	High (requires complete methodology and passing review)
Flexibility	Higher (deviations require disclosure but don't block submission)	Lower (major deviations may affect acceptance)

Registered Reports Process

Stage 1: Submit research plan
├── Introduction (theoretical background, literature review)
├── Methods (complete methodology, analysis plan)
├── Pilot data (if available)
└── Interpretation plan for predicted results
         ↓
Stage 1 Review (research design quality)
├── Accept (In-Principle Acceptance, IPA)
├── Revise and resubmit
└── Reject
         ↓
Stage 2: Conduct research, write results
├── Strictly follow the Stage 1 plan
├── Report all preregistered analyses (including null results)
├── Exploratory analyses clearly labeled
└── Deviations disclosed and explained
         ↓
Stage 2 Review (execution quality)
├── Was the Stage 1 plan faithfully executed?
├── Are results reported completely?
└── Typically not rejected due to null results
         ↓
Publication

Selected Higher Education Journals Supporting Registered Reports

• Studies in Higher Education
• Higher Education
• Assessment & Evaluation in Higher Education
• Teaching in Higher Education
• Educational Research Review
• Learning and Instruction

Full list: COS Registered Reports

---

Quick Reference: 3 Steps to Preregistration

1. Decide whether to preregister: Determine if your research involves hypothesis testing
2. Choose a platform: Use PROSPERO for systematic reviews, OSF for everything else
3. Fill in the 21-item checklist: Use the templates/preregistration_template.md template

Preregistration is not a perfect solution, but it is currently the most practical transparency tool. Even an imperfect preregistration is better than no preregistration at all.