How to Improve Competitive Intelligence Predictive Accuracy: Delphi Panels
A number of years ago, two competitive intelligence researchers at The Rand Corporation developed a technique called the Delphi method to help achieve higher accuracy in predicting futures that involve high levels of ambiguity. Nobody really likes to define the Delphi method because it exists in numerous forms, but the process usually involves a moderator (you, for example) through whom a panel of experts communicates.
When conducting Delphi panels, don’t let the experts communicate directly with one another. By acting as the gatekeeper, you can protect the anonymity of participants and prevent any of the more assertive participants from swaying the group’s opinion. After completing separate interviews for the Delphi panel, you may want to add a group interview.
By introducing group dynamics, you can often develop a deeper understanding of the issues and maybe even gain a surprise benefit from the impromptu nature of the conversation.
The process goes something like this:
Send a questionnaire about a future issue (or issues) to six or so experts and collect feedback from all participants.
Don’t introduce your own opinions or bias into the questionnaire.
Compile a summary of the responses that indicates areas of agreement and disagreement, making sure not to disclose anyone’s identity.
Communicate your findings to all the experts and request additional feedback.
Explore disagreements among the experts; don’t ignore them. If you dismiss the opinions of certain experts, they’re likely to drop out prior to completing the process.
Repeat Steps 2 and 3 to move the experts closer and closer to consensus until you finally achieve general agreement about the future issue(s) in question.
Step 4 has a lot of variations. Just keep in mind the goal: to facilitate movement to consensus about the future issue(s).
As most people find who work with the Delphi method, getting a group of experts to move toward agreement about an issue is very challenging. In some cases, the problem can be bias. In others, lack of knowledge among one or more participants may play a role. Sometimes experts are knowledgeable in current issues but uninformed about future developments in their field. How do you deal with that?
My solution has resulted in modified Delphi panels. Here’s what you do:
Overpopulate your group of experts.
Instead of the traditional 6 experts that many people use, begin with 10 to 12.
Conduct separate interviews with all the experts on your panel via questionnaires, the telephone, or videoconferencing.
Evaluate the experts’ responses.
As you look over the responses, you’ll find that certain experts reveal themselves as the outliers — they don’t seem to understand the reality of a specific issue, they have a strong bias, or they’re simply unqualified to answer questions in a specific area.
Eliminate the biased or unqualified experts from your final consideration.
Summarize the responses you received based on feedback from the experts who passed your competency evaluation.
Modified Delphi panels work. Hundreds of studies and postmortems (evaluations done years after the work was completed) reveal that the approach is incredibly accurate on a consistent basis.
If you’re interested in finding out more about the Delphi method, check out The Delphi Method: Techniques and Applications, edited by Harold A. Linstone and Murray Turoff. You can download a free copy of the book as a PDF.