Delphi

Delphi is an expert survey to assess long-term issues.  The survey takes place in two or more rounds, enabling participants to assess the same matter multiple times based on the results of previous rounds.

When to use it?
Look: to estimate the likelihood, outcomes, or underlying influences of future events.

Frame: to build shared understanding of long-term issues by giving participants the opportunity to modify their answers through multiple rounds.

How to do it?

  1. Determine a set of questions and a format for answers (open ended, multiple choice, etc).

  2. Enlist a panel of experts.

  3. Circulate the questions to each expert, who answers anonymously.

  4. Aggregate and analyze the responses through statistical analysis (e.g., ranking, average, and standard deviations of answers) or qualitative clustering of the answers.

  5. Share the results of the first round with the experts and ask them to reconsider their answers based on these results.  Repeat further rounds if necessary.

  6. Draw up a final report of the findings.

Requirements
People:
The group of experts enlisted for the exercise could range from 10-15 to hundred or more, depending on the number of topics, expected participation rate, and scale of the exercise (e.g., local, national, or international).

Time:
A small survey might require 1 or 2 weeks, while larger surveys might take up to several months.

Pros and Cons
Pros:

  • Delphi works well with dispersed experts to increase original ideas.
  • Anonymity can help reduce groupthink by allowing experts to express their opinions freely.

Cons:

  • Some experts could rate the importance of their own research specialty significantly higher than other experts.
  • The lack of dialogue among participants could leave experts entrenched in their opinions, even after several rounds.

Example
The Millennium Project runs real-time Delphi processes on a number of topics, including environmental security and global ethical issues.  Unlike traditional Delphi processes, which take weeks or months, real-time Delphi records participants’ answers in real time and allows experts to revisit the questionnaire as many times as they want.

Additional Resources

Cuhls, Kerstin. (undated). Delphi method.  Retrieved from: http://www.unido.org/fileadmin/import/16959_DelphiMethod.pdf.

Helmer, Olaf. (1967). Analysis of the future: the Delphi method.  The Rand Corporation. Retrieved from: http://www.rand.org/pubs/papers/P3558.html

The Millennium Project. (ongoing). Real-time Delphi. Retrieved from: http://millennium-project.org/millennium/RTD-general.html

UK Cabinet Office & Government Office for Science. (2014). Futures toolkit: tools for strategic futures for policy-makers and analysts. Retrieved from: https://www.gov.uk/government/publications/futures-toolkit-for-policy-makers-and-analysts

​