Given a fixed budget, this tool determines the optimal number of forecasting questions to include in a study to maximize power to detect differences between groups.
Key Assumptions
- Assumes equal allocation of participants to each condition (i.e., equally powered pairwise comparisons)
- Assumes within-participant correlation in forecasting accuracy (ρ) is constant across all question pairs
- Assumes aggregated scores are approximately normally distributed
- Outputs reflect 80% power to detect differences between groups, two-tailed α = 0.05
Total amount available for participant compensation
Default 0.10 based on FRI pilot studies (range: 0.06-0.12)
Anticipated time participants will spend on each question. Default 2.5 min for quantile elicitation questions
Anticipated time for instructions, consent, demographics, etc. - everything other than the forecasting questions.
Default $0.20/min on Prolific/CloudResearch (incl. fees)
Total conditions including control. Assumes random allocation to power all pairwise comparisons equally.
Results
Optimal Number of Questions
-
Minimum Detectable Effect
-
Cohen's d (80% power, α = 0.05)
Sample Size
-