Delphi is an expert survey to assess long-term issues. The survey takes
place in two or more rounds, enabling participants to assess the same matter
multiple times based on the results of previous rounds.
When to use it?
Look: to estimate the likelihood, outcomes,
or underlying influences of future events.
Frame: to build shared understanding of long-term issues by giving
participants the opportunity to modify their answers through multiple
How to do it?
Determine a set of questions and a format for answers (open ended, multiple
Enlist a panel of experts.
Circulate the questions to each expert, who answers anonymously.
Aggregate and analyze the responses through statistical analysis (e.g.,
ranking, average, and standard deviations of answers) or qualitative clustering
of the answers.
Share the results of the first round with the experts and ask them to
reconsider their answers based on these results. Repeat further rounds if
Draw up a final report of the findings.
The group of experts enlisted for
the exercise could range from 10-15 to hundred or more, depending on the number
of topics, expected participation rate, and scale of the exercise (e.g., local,
national, or international).
A small survey might require 1 or 2 weeks, while larger
surveys might take up to several months.
Pros and Cons
- Delphi works well with dispersed experts to increase original ideas.
- Anonymity can help reduce groupthink by allowing experts to express their opinions freely.
- Some experts could rate the importance of their own research specialty significantly higher than other experts.
- The lack of dialogue among participants could leave experts entrenched in their opinions, even after several rounds.
The Millennium Project runs real-time Delphi processes on a
number of topics, including environmental security and global ethical issues.
Unlike traditional Delphi processes, which take weeks or months, real-time
Delphi records participants’ answers in real time and allows experts to revisit
the questionnaire as many times as they want.
Cuhls, Kerstin. (undated). Delphi method. Retrieved from: http://www.unido.org/fileadmin/import/16959_DelphiMethod.pdf.
Helmer, Olaf. (1967). Analysis of the future: the Delphi method. The
Rand Corporation. Retrieved from: http://www.rand.org/pubs/papers/P3558.html
The Millennium Project. (ongoing). Real-time Delphi. Retrieved from:
UK Cabinet Office & Government Office for Science. (2014). Futures
toolkit: tools for strategic futures for policy-makers and analysts.
Retrieved from: https://www.gov.uk/government/publications/futures-toolkit-for-policy-makers-and-analysts